You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/02/28 19:49:23 UTC

[JENKINS] Lucene » Lucene-Solr-NightlyTests-8.x - Build # 196 - Still Failing!

Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-NightlyTests-8.x/196/

2 tests failed.
FAILED:  org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest.test

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([EF05E42A6C90C4AC]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([EF05E42A6C90C4AC]:0)




Build Log:
[...truncated 16404 lines...]
   [junit4] Suite: org.apache.solr.cloud.ChaosMonkeyNothingIsSafeTest
   [junit4]   2> 1671007 INFO  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[EF05E42A6C90C4AC]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 1671007 INFO  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[EF05E42A6C90C4AC]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1671008 INFO  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[EF05E42A6C90C4AC]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/data-dir-137-001
   [junit4]   2> 1671008 WARN  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[EF05E42A6C90C4AC]-worker) [     ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=10 numCloses=10
   [junit4]   2> 1671008 INFO  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[EF05E42A6C90C4AC]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1671010 INFO  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[EF05E42A6C90C4AC]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
   [junit4]   2> 1671010 INFO  (SUITE-ChaosMonkeyNothingIsSafeTest-seed#[EF05E42A6C90C4AC]-worker) [     ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 1671014 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1671015 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 1671015 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1671019 WARN  (ZkTestServer Run Thread) [     ] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0.
   [junit4]   2> 1671115 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer start zk server on port: 39087
   [junit4]   2> 1671115 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:39087
   [junit4]   2> 1671115 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:39087
   [junit4]   2> 1671115 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 39087
   [junit4]   2> 1671116 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1671119 INFO  (zkConnectionManagerCallback-13703-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1671119 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1671121 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1671122 INFO  (zkConnectionManagerCallback-13705-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1671122 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1671123 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 1671125 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 1671127 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 1671128 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 1671129 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 1671130 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 1671130 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 1671131 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 1671132 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 1671133 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 1671134 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 1671135 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 1671240 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1671240 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1671241 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 1671241 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1671241 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1671241 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1671241 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@695eba86{/,null,AVAILABLE}
   [junit4]   2> 1671242 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@4ced54f4{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:36269}
   [junit4]   2> 1671242 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.e.j.s.Server Started @1671273ms
   [junit4]   2> 1671242 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=36269, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/control-001/cores}
   [junit4]   2> 1671242 ERROR (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1671242 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1671242 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 1671242 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1671242 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 1671242 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-02-28T17:30:50.272Z
   [junit4]   2> 1671243 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1671244 INFO  (zkConnectionManagerCallback-13707-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1671244 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1671345 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1671345 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/control-001/solr.xml
   [junit4]   2> 1671348 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1671348 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1671349 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1671618 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 1671619 WARN  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@54cef619[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1671619 WARN  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@54cef619[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1671622 WARN  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@21c7a02c[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1671622 WARN  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@21c7a02c[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1671624 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39087/solr
   [junit4]   2> 1671624 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1671625 INFO  (zkConnectionManagerCallback-13718-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1671625 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1671626 WARN  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]-SendThread(127.0.0.1:39087)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103a09e7e770003.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103a09e7e770003, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103a09e7e770003, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 1671728 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1671729 INFO  (zkConnectionManagerCallback-13720-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1671729 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1671895 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36269_
   [junit4]   2> 1671896 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.c.Overseer Overseer (id=73078621555064836-127.0.0.1:36269_-n_0000000000) starting
   [junit4]   2> 1671913 INFO  (OverseerStateUpdate-73078621555064836-127.0.0.1:36269_-n_0000000000) [n:127.0.0.1:36269_     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:36269_
   [junit4]   2> 1671914 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36269_
   [junit4]   2> 1671917 INFO  (OverseerStateUpdate-73078621555064836-127.0.0.1:36269_-n_0000000000) [n:127.0.0.1:36269_     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1671925 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1671925 WARN  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1671937 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1671978 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1671985 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1671985 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1671986 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [n:127.0.0.1:36269_     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/control-001/cores
   [junit4]   2> 1672002 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1672006 INFO  (zkConnectionManagerCallback-13737-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1672006 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1672010 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1672013 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39087/solr ready
   [junit4]   2> 1672022 INFO  (OverseerThreadFactory-13727-thread-1-processing-n:127.0.0.1:36269_) [n:127.0.0.1:36269_     ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 1672128 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_    x:control_collection_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1672129 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_    x:control_collection_shard1_replica_n1 ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1673155 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 1673168 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 1673172 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1673187 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from configset conf1, trusted=true
   [junit4]   2> 1673187 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1673187 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/control-001/cores/control_collection_shard1_replica_n1/data/]
   [junit4]   2> 1673189 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=8, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1673196 WARN  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1673240 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1673240 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1673241 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 15000ms; 
   [junit4]   2> 1673241 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1673242 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=1.1279296875, floorSegmentMB=1.990234375, forceMergeDeletesPctAllowed=28.332362413321142, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=27.859921062874125
   [junit4]   2> 1673245 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1673245 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1673245 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1673246 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1692961029253758976
   [junit4]   2> 1673249 INFO  (searcherExecutor-13739-thread-1-processing-n:127.0.0.1:36269_ x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [control_collection_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1673252 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 1673252 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/control_collection/leaders/shard1
   [junit4]   2> 1673255 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1673255 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1673255 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36269/control_collection_shard1_replica_n1/
   [junit4]   2> 1673256 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1673256 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy http://127.0.0.1:36269/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 1673256 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/73078621555064836-core_node2-n_0000000000
   [junit4]   2> 1673259 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36269/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 1673361 INFO  (zkCallback-13719-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1673361 INFO  (zkCallback-13719-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1673362 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_ c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1673365 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1236
   [junit4]   2> 1673367 INFO  (qtp1803164463-88825) [n:127.0.0.1:36269_     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 1673464 INFO  (zkCallback-13719-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1673464 INFO  (zkCallback-13719-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1673465 INFO  (zkCallback-13719-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1673465 INFO  (qtp1803164463-88825) [n:127.0.0.1:36269_     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:36269_&wt=javabin&version=2} status=0 QTime=1450
   [junit4]   2> 1673465 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Waiting to see 1 active replicas in collection: control_collection
   [junit4]   2> 1673571 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1673572 INFO  (zkConnectionManagerCallback-13748-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1673572 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1673573 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1673574 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39087/solr ready
   [junit4]   2> 1673574 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 1673577 INFO  (OverseerThreadFactory-13727-thread-2-processing-n:127.0.0.1:36269_) [n:127.0.0.1:36269_     ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 1673578 INFO  (OverseerCollectionConfigSetProcessor-73078621555064836-127.0.0.1:36269_-n_0000000000) [n:127.0.0.1:36269_     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1673781 WARN  (OverseerThreadFactory-13727-thread-2-processing-n:127.0.0.1:36269_) [n:127.0.0.1:36269_     ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 1673783 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 1673786 INFO  (qtp1803164463-88827) [n:127.0.0.1:36269_     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=210
   [junit4]   2> 1673787 INFO  (watches-13745-thread-1) [     ] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1
   [junit4]   2> 1673787 INFO  (watches-13745-thread-1) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 1673788 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1
   [junit4]   2> 1673788 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 1673788 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1
   [junit4]   2> 1673788 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 1673789 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1
   [junit4]   2> 1673789 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 1673789 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Creating jetty instances pullReplicaCount=0 numOtherReplicas=4
   [junit4]   2> 1673896 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-1-001 of type NRT for shard1
   [junit4]   2> 1673896 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1673897 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1673897 INFO  (closeThreadPool-13749-thread-1) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 1673897 INFO  (closeThreadPool-13749-thread-1) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1673898 INFO  (closeThreadPool-13749-thread-1) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1673898 INFO  (closeThreadPool-13749-thread-1) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1673898 INFO  (closeThreadPool-13749-thread-1) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@442d5a09{/,null,AVAILABLE}
   [junit4]   2> 1673900 INFO  (closeThreadPool-13749-thread-1) [     ] o.e.j.s.AbstractConnector Started ServerConnector@21f29470{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:44085}
   [junit4]   2> 1673900 INFO  (closeThreadPool-13749-thread-1) [     ] o.e.j.s.Server Started @1673931ms
   [junit4]   2> 1673900 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=44085, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-1-001/cores}
   [junit4]   2> 1673900 ERROR (closeThreadPool-13749-thread-1) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1673900 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1673900 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 1673900 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1673900 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 1673900 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-02-28T17:30:52.930Z
   [junit4]   2> 1673902 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1673904 INFO  (zkConnectionManagerCallback-13751-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1673904 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1674005 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-2-001 of type NRT for shard1
   [junit4]   2> 1674006 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1674006 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1674006 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1674006 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-1-001/solr.xml
   [junit4]   2> 1674006 INFO  (closeThreadPool-13749-thread-2) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 1674008 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1674008 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1674027 INFO  (closeThreadPool-13749-thread-2) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1674027 INFO  (closeThreadPool-13749-thread-2) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1674027 INFO  (closeThreadPool-13749-thread-2) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1674027 INFO  (closeThreadPool-13749-thread-2) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@39e24c4{/,null,AVAILABLE}
   [junit4]   2> 1674027 INFO  (closeThreadPool-13749-thread-2) [     ] o.e.j.s.AbstractConnector Started ServerConnector@3fb188c1{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:34207}
   [junit4]   2> 1674027 INFO  (closeThreadPool-13749-thread-2) [     ] o.e.j.s.Server Started @1674058ms
   [junit4]   2> 1674027 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=34207, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-2-001/cores}
   [junit4]   2> 1674028 ERROR (closeThreadPool-13749-thread-2) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1674028 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1674028 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 1674028 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1674028 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 1674028 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-02-28T17:30:53.058Z
   [junit4]   2> 1674028 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1674029 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1674032 INFO  (zkConnectionManagerCallback-13753-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1674032 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1674128 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-3-001 of type NRT for shard1
   [junit4]   2> 1674128 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1674128 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1674129 INFO  (closeThreadPool-13749-thread-3) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 1674133 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1674133 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-2-001/solr.xml
   [junit4]   2> 1674135 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1674135 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1674136 INFO  (closeThreadPool-13749-thread-3) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1674136 INFO  (closeThreadPool-13749-thread-3) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1674136 INFO  (closeThreadPool-13749-thread-3) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1674137 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1674140 INFO  (closeThreadPool-13749-thread-3) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7ae9c71b{/,null,AVAILABLE}
   [junit4]   2> 1674140 INFO  (closeThreadPool-13749-thread-3) [     ] o.e.j.s.AbstractConnector Started ServerConnector@2d2fc72f{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:46865}
   [junit4]   2> 1674140 INFO  (closeThreadPool-13749-thread-3) [     ] o.e.j.s.Server Started @1674171ms
   [junit4]   2> 1674140 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=46865, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-3-001/cores}
   [junit4]   2> 1674141 ERROR (closeThreadPool-13749-thread-3) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1674141 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1674141 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 1674141 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1674141 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 1674141 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-02-28T17:30:53.171Z
   [junit4]   2> 1674145 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1674145 INFO  (zkConnectionManagerCallback-13761-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1674145 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1674246 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1674246 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-3-001/solr.xml
   [junit4]   2> 1674249 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1674249 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1674249 INFO  (TEST-ChaosMonkeyNothingIsSafeTest.test-seed#[EF05E42A6C90C4AC]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-4-001 of type NRT for shard1
   [junit4]   2> 1674250 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1674252 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1674252 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1674253 INFO  (closeThreadPool-13749-thread-4) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 1674296 INFO  (closeThreadPool-13749-thread-4) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1674296 INFO  (closeThreadPool-13749-thread-4) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1674296 INFO  (closeThreadPool-13749-thread-4) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1674300 INFO  (closeThreadPool-13749-thread-4) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7b2b0ffc{/,null,AVAILABLE}
   [junit4]   2> 1674300 INFO  (closeThreadPool-13749-thread-4) [     ] o.e.j.s.AbstractConnector Started ServerConnector@74f4e42b{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:33353}
   [junit4]   2> 1674300 INFO  (closeThreadPool-13749-thread-4) [     ] o.e.j.s.Server Started @1674331ms
   [junit4]   2> 1674300 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/tempDir-001/jetty4, solrconfig=solrconfig.xml, hostContext=/, hostPort=33353, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-4-001/cores}
   [junit4]   2> 1674301 ERROR (closeThreadPool-13749-thread-4) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1674301 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1674301 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 1674301 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1674301 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 1674301 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-02-28T17:30:53.331Z
   [junit4]   2> 1674308 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1674316 INFO  (zkConnectionManagerCallback-13766-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1674316 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1674417 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 1674417 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-4-001/solr.xml
   [junit4]   2> 1674419 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1674419 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1674421 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1674512 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 1674513 WARN  (closeThreadPool-13749-thread-3) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@1a4226de[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674513 WARN  (closeThreadPool-13749-thread-3) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@1a4226de[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674516 WARN  (closeThreadPool-13749-thread-3) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@1d78c4d4[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674516 WARN  (closeThreadPool-13749-thread-3) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@1d78c4d4[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674517 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39087/solr
   [junit4]   2> 1674535 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1674551 INFO  (zkConnectionManagerCallback-13777-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1674551 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1674660 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1674665 INFO  (zkConnectionManagerCallback-13779-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1674665 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1674672 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1674675 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.c.ZkController Publish node=127.0.0.1:46865_ as DOWN
   [junit4]   2> 1674676 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1674676 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46865_
   [junit4]   2> 1674677 INFO  (zkCallback-13719-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1674678 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1674678 WARN  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1674687 INFO  (zkCallback-13747-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1674692 INFO  (zkCallback-13778-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1674700 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1674720 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1674731 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1674731 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1674733 INFO  (closeThreadPool-13749-thread-3) [n:127.0.0.1:46865_     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-3-001/cores
   [junit4]   2> 1674761 INFO  (closeThreadPool-13749-thread-3) [     ] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:46865_
   [junit4]   2> 1674795 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 1674800 WARN  (closeThreadPool-13749-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@1cf4845b[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674800 WARN  (closeThreadPool-13749-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@1cf4845b[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674802 WARN  (closeThreadPool-13749-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@60cba9d5[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674802 WARN  (closeThreadPool-13749-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@60cba9d5[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674803 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39087/solr
   [junit4]   2> 1674806 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1674807 INFO  (zkConnectionManagerCallback-13795-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1674807 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1674911 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1674911 INFO  (zkConnectionManagerCallback-13797-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1674911 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1674915 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1674917 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.c.ZkController Publish node=127.0.0.1:44085_ as DOWN
   [junit4]   2> 1674918 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1674918 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44085_
   [junit4]   2> 1674920 INFO  (zkCallback-13747-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1674920 INFO  (zkCallback-13778-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1674920 INFO  (zkCallback-13719-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1674922 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1674922 WARN  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1674930 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 1674930 WARN  (closeThreadPool-13749-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@4312083a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674930 WARN  (closeThreadPool-13749-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@4312083a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674932 WARN  (closeThreadPool-13749-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3863271e[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674932 WARN  (closeThreadPool-13749-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3863271e[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1674934 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39087/solr
   [junit4]   2> 1674934 INFO  (zkCallback-13796-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1674936 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1674944 INFO  (zkConnectionManagerCallback-13809-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1674944 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1674948 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1674970 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1674980 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1674981 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1674982 INFO  (closeThreadPool-13749-thread-1) [n:127.0.0.1:44085_     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-1-001/cores
   [junit4]   2> 1675003 INFO  (closeThreadPool-13749-thread-1) [     ] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:44085_
   [junit4]   2> 1675049 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1675050 INFO  (zkConnectionManagerCallback-13815-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1675050 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1675053 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1675056 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.c.ZkController Publish node=127.0.0.1:34207_ as DOWN
   [junit4]   2> 1675057 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1675057 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34207_
   [junit4]   2> 1675060 INFO  (zkCallback-13778-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1675060 INFO  (zkCallback-13719-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1675060 INFO  (zkCallback-13796-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1675060 INFO  (zkCallback-13747-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1675061 INFO  (zkCallback-13814-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1675062 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1675062 WARN  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1675076 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1675099 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1675109 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1675109 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1675111 INFO  (closeThreadPool-13749-thread-2) [n:127.0.0.1:34207_     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-2-001/cores
   [junit4]   2> 1675153 INFO  (closeThreadPool-13749-thread-2) [     ] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:34207_
   [junit4]   2> 1675579 INFO  (OverseerCollectionConfigSetProcessor-73078621555064836-127.0.0.1:36269_-n_0000000000) [n:127.0.0.1:36269_     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1675682 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 1675682 WARN  (closeThreadPool-13749-thread-4) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@12f0afcf[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1675682 WARN  (closeThreadPool-13749-thread-4) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@12f0afcf[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1675685 WARN  (closeThreadPool-13749-thread-4) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@6c23f06d[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1675685 WARN  (closeThreadPool-13749-thread-4) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@6c23f06d[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1675686 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39087/solr
   [junit4]   2> 1675687 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1675687 INFO  (zkConnectionManagerCallback-13831-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1675687 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1675789 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1675790 INFO  (zkConnectionManagerCallback-13833-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1675790 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1675794 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1675797 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.c.ZkController Publish node=127.0.0.1:33353_ as DOWN
   [junit4]   2> 1675798 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1675798 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33353_
   [junit4]   2> 1675799 INFO  (zkCallback-13796-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1675800 INFO  (zkCallback-13747-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1675800 INFO  (zkCallback-13778-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1675800 INFO  (zkCallback-13814-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1675800 INFO  (zkCallback-13719-thread-3) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1675800 INFO  (zkCallback-13832-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
   [junit4]   2> 1675801 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1675801 WARN  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1675816 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1675836 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1675846 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1675846 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1675847 INFO  (closeThreadPool-13749-thread-4) [n:127.0.0.1:33353_     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-4-001/cores
   [junit4]   2> 1675871 INFO  (closeThreadPool-13749-thread-4) [     ] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:33353_
   [junit4]   2> 1675882 INFO  (OverseerThreadFactory-13727-thread-3-processing-n:127.0.0.1:36269_) [n:127.0.0.1:36269_ c:collection1 s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:34207_ for creating new replica of shard shard1 for collection collection1
   [junit4]   2> 1675883 INFO  (OverseerThreadFactory-13727-thread-3-processing-n:127.0.0.1:36269_) [n:127.0.0.1:36269_ c:collection1 s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 1675888 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_    x:collection1_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1676899 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 1676925 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 1676929 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1676943 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from configset conf1, trusted=true
   [junit4]   2> 1676944 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1676944 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-2-001/cores/collection1_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.ChaosMonkeyNothingIsSafeTest_EF05E42A6C90C4AC-001/shard-2-001/cores/collection1_shard1_replica_n1/data/]
   [junit4]   2> 1676946 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=8, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 1676957 WARN  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 1676997 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1676998 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1676999 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 15000ms; 
   [junit4]   2> 1676999 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1677000 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=27, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=1.1279296875, floorSegmentMB=1.990234375, forceMergeDeletesPctAllowed=28.332362413321142, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=27.859921062874125
   [junit4]   2> 1677003 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1677003 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1677003 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1677003 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1692961033193259008
   [junit4]   2> 1677007 INFO  (searcherExecutor-13844-thread-1-processing-n:127.0.0.1:34207_ x:collection1_shard1_replica_n1 c:collection1 s:shard1) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [collection1_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1677010 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 1677010 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
   [junit4]   2> 1677015 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1677015 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1677015 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34207/collection1_shard1_replica_n1/
   [junit4]   2> 1677016 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1677016 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy http://127.0.0.1:34207/collection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 1677016 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/73078621555064848-core_node2-n_0000000000
   [junit4]   2> 1677019 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34207/collection1_shard1_replica_n1/ shard1
   [junit4]   2> 1677121 INFO  (zkCallback-13814-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1677121 INFO  (zkCallback-13814-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1677123 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_ c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1677125 INFO  (qtp999164776-88897) [n:127.0.0.1:34207_     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1237
   [junit4]   2> 1677129 INFO  (qtp1892384490-88919) [n:127.0.0.1:33353_ c:collection1    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:34207_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1254
   [junit4]   2> 1677225 INFO  (zkCallback-13814-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1677225 INFO  (zkCallback-13814-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1677225 INFO  (zkCallback-13814-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1677882 INFO  (OverseerCollectionConfigSetProcessor-73078621555064836-127.0.0.1:36269_-n_0000000000) [n:127.0.0.1:36269_     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1677885 INFO  (OverseerThreadFactory-13727-thread-4-processing-n:127.0.0.1:36269_) [n:127.0.0.1:36269_ c:collection1 s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:44085_ for creating new replica of shard shard1 for collection collection1
   [junit4]   2> 1677890 INFO  (OverseerThreadFactory-13727-thread-4-processing-n:127.0.0.1:36269_) [n:127.0.0.1:36269_ c:collection1 s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 1677894 INFO  (qtp634192913-88883) [n:127.0.0.1:44085_    x:collection1_shard1_replica_n3 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n3&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1678005 INFO  (zkCallback-13814-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1678005 INFO  (zkCallback-13814-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1678005 INFO  (zkCallback-13814-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
   [junit4]   2> 1678906 INFO  (qtp634192913-88883) [n:127.0.0.1:44085_ c:collection1 s:shard1  x:collection1_shard1_replica_n3 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 1678931 INFO  (qtp634192913-88883) [n:127.0.0.1:44085_ c:collection1 s:shard1  x:collection1_shard1_replica_n3 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 1678935 INFO  (qtp634192913-88883) [n:127.0.0.1:44085_ c:collection1 s:shard1  x:collection1_shard1_replica_n3 ] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 1678973 INFO  (qtp634192913-88883) [n:127.0.0.1:44085_ c:collection1 s:shard1  x:collection1_shard1_replica_n3 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n3' using configuration from configset conf1, trusted=true
   [junit4]   2> 1678974 INFO  (qtp634192913-88883) [n:127.0.0.1:44085_ c:collection1 s:shard1  x:collection1_shard1_replica_n3 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n3' (registry 'solr.core.collection1.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3cc5d6c6
   [junit4]   2> 1678974 INFO  (qtp634192913-88883) [n:127.0.0.1:44085_ c:collection1 s:shard1  x:collection1_shard1_replica_n3 ] o.a.s.c.SolrCore [[collection1_shard1_replica_n3] ] Opening new SolrC

[...truncated too long message...]

       at org.eclipse.jetty.io.ManagedSelector.nioSelect(ManagedSelector.java:149)
   [junit4]   2>         at org.eclipse.jetty.io.ManagedSelector.select(ManagedSelector.java:156)
   [junit4]   2>         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:572)
   [junit4]   2>         at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:509)
   [junit4]   2>         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produceTask(EatWhatYouKill.java:360)
   [junit4]   2>         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:184)
   [junit4]   2>         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
   [junit4]   2>         at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:135)
   [junit4]   2>         at org.eclipse.jetty.io.ManagedSelector$$Lambda$96/1569224959.run(Unknown Source)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
   [junit4]   2>         at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>   136) Thread[id=89247, name=Session-HouseKeeper-75aacec5-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2>   137) Thread[id=88859, name=Connector-Scheduler-4ced54f4-1, state=TIMED_WAITING, group=TGRP-ChaosMonkeyNothingIsSafeTest]
   [junit4]   2>         at sun.misc.Unsafe.park(Native Method)
   [junit4]   2>         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2>         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
   [junit4]   2>         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
   [junit4]   2>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
   [junit4]   2>         at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene87): {rnd_b=PostingsFormat(name=Direct), a_t=BlockTreeOrds(blocksize=128), _root_=Lucene84, id=PostingsFormat(name=Direct)}, docValues:{_version_=DocValuesFormat(name=Lucene80), a_i=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=952, maxMBSortInHeap=7.397340956572851, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=id, timezone=America/Vancouver
   [junit4]   2> NOTE: Linux 5.4.0-45-generic amd64/Oracle Corporation 1.8.0_281 (64-bit)/cpus=4,threads=154,free=243427976,total=473956352
   [junit4]   2> NOTE: All tests run in this JVM: [ConvertedLegacyTest, DistributedQueryElevationComponentTest, SuggesterTSTTest, TestHalfAndHalfDocValues, TestBlockCollapse, TestPHPSerializedResponseWriter, PreAnalyzedFieldManagedSchemaCloudTest, TestWordDelimiterFilterFactory, CoreAdminOperationTest, BlockDirectoryTest, TestReplicationHandlerBackup, TestTlogReplayVsRecovery, JSONWriterTest, DistributedFacetPivotWhiteBoxTest, TestDynamicLoadingUrl, JWTAuthPluginTest, ConcurrentCreateRoutedAliasTest, BadComponentTest, TestClusterStateMutator, HdfsChaosMonkeyNothingIsSafeTest, CollectionsAPIDistributedZkTest, CdcrOpsAndBoundariesTest, DirectoryFactoryTest, TestConfigSetsAPI, TestDownShardTolerantSearch, NotRequiredUniqueKeyTest, RootFieldTest, BinaryUpdateRequestHandlerTest, DistributedFacetPivotLargeTest, TestIBSimilarityFactory, PeerSyncReplicationTest, SpatialHeatmapFacetsTest, TestUtilizeNode, DeleteShardTest, FacetPivotSmallTest, CachingDirectoryFactoryTest, TestHashQParserPlugin, SolrIndexSearcherTest, TestUnInvertedFieldException, TestSimPolicyCloud, TestMaxTokenLenTokenizer, TestOnReconnectListenerSupport, TestApiFramework, TestBooleanSimilarityFactory, HttpSolrCallGetCoreTest, TestQueryingOnDownCollection, ImplicitSnitchTest, TestQueryUtils, TestCloudInspectUtil, TestDocTermOrds, TestReload, ComputePlanActionTest, FieldMutatingUpdateProcessorTest, TestSolrCLIRunExample, TestUtils, SolrInfoBeanTest, ExternalFileFieldSortTest, OverseerStatusTest, ClassificationUpdateProcessorFactoryTest, TestFilteredDocIdSet, HdfsRecoveryZkTest, DistanceFunctionTest, TestNonDefinedSimilarityFactory, TestInitParams, TestFunctionQuery, TestCollectionsAPIViaSolrCloudCluster, TestConfigSetImmutable, TestManagedResource, CurrencyRangeFacetCloudTest, NestedAtomicUpdateTest, IndexBasedSpellCheckerTest, TestSolrCloudWithSecureImpersonation, QueryResultKeyTest, PeerSyncWithBufferUpdatesTest, QueryElevationComponentTest, HdfsTlogReplayBufferedWhileIndexingTest, TestFaceting, SolrRequestParserTest, ClusterStateUpdateTest, TestSurroundQueryParser, TestSolrCachePerf, DocValuesMissingTest, TlogReplayBufferedWhileIndexingTest, SearchHandlerTest, PKIAuthenticationIntegrationTest, SolrMetricManagerTest, AddBlockUpdateTest, TestSolrQueryParser, TestFuzzyAnalyzedSuggestions, CopyFieldTest, FullHLLTest, SolrShardReporterTest, SpatialRPTFieldTypeTest, TestEmbeddedSolrServerConstructors, AuthToolTest, V2ApiIntegrationTest, AdminHandlersProxyTest, ZkCLITest, RegexBytesRefFilterTest, TestTolerantSearch, TestStressRecovery, TestFieldCacheSort, TestDocumentBuilder, TestCopyFieldCollectionResource, XmlInterpolationTest, TestExactStatsCache, TestSolrCloudWithHadoopAuthPlugin, SystemCollectionCompatTest, ScheduledTriggerIntegrationTest, StatelessScriptUpdateProcessorFactoryTest, ActionThrottleTest, TestNumericRangeQuery32, TestUnifiedSolrHighlighterWithoutStoredId, TestLRUCache, TestRandomDVFaceting, AssignBackwardCompatibilityTest, BasicZkTest, ChaosMonkeyNothingIsSafeTest]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=ChaosMonkeyNothingIsSafeTest -Dtests.seed=EF05E42A6C90C4AC -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=id -Dtests.timezone=America/Vancouver -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J2 | ChaosMonkeyNothingIsSafeTest (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([EF05E42A6C90C4AC]:0)
   [junit4] Completed [934/934 (1!)] on J2 in 7213.96s, 1 test, 2 errors <<< FAILURES!

[...truncated 50732 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:652: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:507: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:494: Source checkout is dirty (unversioned/missing files) after running tests!!! Offending files:
* solr/contrib/analytics/src/test-files/solr/filestore
* solr/contrib/analytics/src/test-files/solr/userfiles
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/collection1/conf/dataimport.properties
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/filestore
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/userfiles
* solr/contrib/dataimporthandler/src/test-files/dih/solr/filestore
* solr/contrib/dataimporthandler/src/test-files/dih/solr/userfiles
* solr/contrib/extraction/src/test-files/extraction/solr/filestore
* solr/contrib/extraction/src/test-files/extraction/solr/userfiles
* solr/contrib/langid/src/test-files/langid/solr/filestore
* solr/contrib/langid/src/test-files/langid/solr/userfiles
* solr/contrib/ltr/src/test-files/solr/filestore
* solr/contrib/ltr/src/test-files/solr/userfiles
* solr/contrib/prometheus-exporter/src/test-files/solr/filestore
* solr/contrib/prometheus-exporter/src/test-files/solr/userfiles
* solr/contrib/velocity/src/test-files/velocity/solr/filestore
* solr/contrib/velocity/src/test-files/velocity/solr/userfiles
* solr/core/src/test-files/solr/filestore
* solr/core/src/test-files/solr/userfiles
* solr/solrj/src/test-files/solrj/solr/multicore/filestore
* solr/solrj/src/test-files/solrj/solr/multicore/userfiles

Total time: 335 minutes 31 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene » Lucene-Solr-NightlyTests-8.x - Build # 206 - Still Unstable!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-NightlyTests-8.x/206/

1 tests failed.
FAILED:  org.apache.solr.util.TestCircuitBreaker.testResponseWithCBTiming

Error Message:
Exception during query

Stack Trace:
java.lang.RuntimeException: Exception during query
	at __randomizedtesting.SeedInfo.seed([98E16660DECD3E21:703DCB822D252401]:0)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:1023)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:983)
	at org.apache.solr.util.TestCircuitBreaker.testResponseWithCBTiming(TestCircuitBreaker.java:245)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.common.SolrException: Circuit Breakers tripped Memory Circuit Breaker triggered as JVM heap usage values are greater than allocated threshold.Seen JVM heap memory usage 446137392 and allocated threshold 395182080

	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:320)
	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:216)
	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2638)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:346)
	at org.apache.solr.util.TestHarness.query(TestHarness.java:328)
	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:997)
	... 45 more




Build Log:
[...truncated 16179 lines...]
   [junit4] Suite: org.apache.solr.util.TestCircuitBreaker
   [junit4]   2> 4706253 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 4706253 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 4706253 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.util.TestCircuitBreaker_98E16660DECD3E21-001/data-dir-294-001
   [junit4]   2> 4706253 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 4706254 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 4706254 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.SolrTestCaseJ4 ####initCore
   [junit4]   2> 4706259 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.c.SolrResourceLoader Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 4706275 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 4706285 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 4706292 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4706542 WARN  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.c.CoreContainer Unable to create [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/userfiles].  Features requiring this directory may fail.
   [junit4]   2>           => java.security.AccessControlException: access denied ("java.io.FilePermission" "/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/userfiles" "write")
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
   [junit4]   2> java.security.AccessControlException: access denied ("java.io.FilePermission" "/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/userfiles" "write")
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_281]
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:886) ~[?:1.8.0_281]
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_281]
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) ~[?:1.8.0_281]
   [junit4]   2> 	at sun.nio.fs.UnixPath.checkWrite(UnixPath.java:801) ~[?:?]
   [junit4]   2> 	at sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:376) ~[?:?]
   [junit4]   2> 	at java.nio.file.Files.createDirectory(Files.java:674) ~[?:1.8.0_281]
   [junit4]   2> 	at java.nio.file.Files.createAndCheckIsDirectory(Files.java:781) ~[?:1.8.0_281]
   [junit4]   2> 	at java.nio.file.Files.createDirectories(Files.java:727) ~[?:1.8.0_281]
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.<init>(CoreContainer.java:377) [java/:?]
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.<init>(CoreContainer.java:342) [java/:?]
   [junit4]   2> 	at org.apache.solr.util.TestHarness.<init>(TestHarness.java:183) [java/:?]
   [junit4]   2> 	at org.apache.solr.util.TestHarness.<init>(TestHarness.java:156) [java/:?]
   [junit4]   2> 	at org.apache.solr.util.TestHarness.<init>(TestHarness.java:162) [java/:?]
   [junit4]   2> 	at org.apache.solr.util.TestHarness.<init>(TestHarness.java:112) [java/:?]
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:812) [java/:?]
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:802) [java/:?]
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:663) [java/:?]
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:652) [java/:?]
   [junit4]   2> 	at org.apache.solr.util.TestCircuitBreaker.setUpClass(TestCircuitBreaker.java:61) [test/:?]
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_281]
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_281]
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_281]
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_281]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.1.jar:4.13.1]
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) [java/:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) [java/:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) [java/:?]
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) [java/:?]
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) [java/:?]
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) [java/:?]
   [junit4]   2> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.1.jar:4.13.1]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_281]
   [junit4]   2> 4706544 WARN  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.f.DistribPackageStore Unable to create [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/filestore] directory in SOLR_HOME [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr].  Features requiring this directory may fail.
   [junit4]   2>           => java.security.AccessControlException: access denied ("java.io.FilePermission" "/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/filestore" "write")
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
   [junit4]   2> java.security.AccessControlException: access denied ("java.io.FilePermission" "/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/filestore" "write")
   [junit4]   2> 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:1.8.0_281]
   [junit4]   2> 	at java.security.AccessController.checkPermission(AccessController.java:886) ~[?:1.8.0_281]
   [junit4]   2> 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549) ~[?:1.8.0_281]
   [junit4]   2> 	at java.lang.SecurityManager.checkWrite(SecurityManager.java:979) ~[?:1.8.0_281]
   [junit4]   2> 	at java.io.File.mkdir(File.java:1311) ~[?:1.8.0_281]
   [junit4]   2> 	at java.io.File.mkdirs(File.java:1343) ~[?:1.8.0_281]
   [junit4]   2> 	at org.apache.solr.filestore.DistribPackageStore.ensurePackageStoreDir(DistribPackageStore.java:561) [java/:?]
   [junit4]   2> 	at org.apache.solr.filestore.DistribPackageStore.<init>(DistribPackageStore.java:81) [java/:?]
   [junit4]   2> 	at org.apache.solr.filestore.PackageStoreAPI.<init>(PackageStoreAPI.java:78) [java/:?]
   [junit4]   2> 	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:678) [java/:?]
   [junit4]   2> 	at org.apache.solr.util.TestHarness.<init>(TestHarness.java:184) [java/:?]
   [junit4]   2> 	at org.apache.solr.util.TestHarness.<init>(TestHarness.java:156) [java/:?]
   [junit4]   2> 	at org.apache.solr.util.TestHarness.<init>(TestHarness.java:162) [java/:?]
   [junit4]   2> 	at org.apache.solr.util.TestHarness.<init>(TestHarness.java:112) [java/:?]
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.createCore(SolrTestCaseJ4.java:812) [java/:?]
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:802) [java/:?]
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:663) [java/:?]
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.initCore(SolrTestCaseJ4.java:652) [java/:?]
   [junit4]   2> 	at org.apache.solr.util.TestCircuitBreaker.setUpClass(TestCircuitBreaker.java:61) [test/:?]
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_281]
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_281]
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_281]
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_281]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.1.jar:4.13.1]
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45) [java/:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41) [java/:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53) [java/:?]
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47) [java/:?]
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64) [java/:?]
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54) [java/:?]
   [junit4]   2> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20) [junit-4.13.1.jar:4.13.1]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368) [randomizedtesting-runner-2.7.2.jar:?]
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748) [?:1.8.0_281]
   [junit4]   2> 4706545 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 4706545 WARN  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3f41f460[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 4706546 WARN  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3f41f460[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 4706548 WARN  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@7b495e89[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 4706548 WARN  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@7b495e89[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 4706550 WARN  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 4706576 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@542842f0
   [junit4]   2> 4706583 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@542842f0
   [junit4]   2> 4706583 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@542842f0
   [junit4]   2> 4706584 INFO  (coreLoadExecutor-32285-thread-1) [     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 4706594 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.c.SolrResourceLoader Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 4706612 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 4706636 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 4706643 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 4706650 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 4706650 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@542842f0
   [junit4]   2> 4706650 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.util.TestCircuitBreaker_98E16660DECD3E21-001/data-dir-294-001/]
   [junit4]   2> 4706651 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=34.2763671875, floorSegmentMB=0.5244140625, forceMergeDeletesPctAllowed=7.733971871504089, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5655964320142205, deletesPctAllowed=26.310028062248033
   [junit4]   2> 4706688 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 4706688 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 4706689 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@13254ed5
   [junit4]   2> 4706689 WARN  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.r.ManagedResourceStorage Cannot write to config directory /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf ; switching to use InMemory storage instead.
   [junit4]   2> 4706691 INFO  (coreLoadExecutor-32285-thread-1) [    x:collection1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 4706692 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.SolrTestCaseJ4 ####initCore end
   [junit4]   2> 4706692 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706692 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706694 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706694 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706695 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706695 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706695 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706704 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706704 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 8
   [junit4]   2> 4706704 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706705 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706705 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706705 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706705 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706706 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706711 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706711 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 5
   [junit4]   2> 4706712 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706712 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706712 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706712 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706713 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706713 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706718 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706718 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706719 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706719 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706719 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706719 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706720 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706720 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706725 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706725 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706726 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706726 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706726 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706726 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706727 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706727 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706733 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706733 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 5
   [junit4]   2> 4706734 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706734 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706734 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706735 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706735 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706735 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706741 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706791 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 55
   [junit4]   2> 4706791 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706792 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706792 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706792 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706793 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706793 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706798 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706798 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706798 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706799 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706799 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706799 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706799 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706800 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706804 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706805 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706805 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706805 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706806 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706806 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706806 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706807 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706815 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706816 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 8
   [junit4]   2> 4706816 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706816 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706817 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706817 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706817 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706818 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706822 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706822 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706823 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706823 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706824 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706824 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706824 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706824 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706843 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706843 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 18
   [junit4]   2> 4706844 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706844 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706844 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706844 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706845 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706845 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706850 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706850 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706851 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706851 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706851 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706852 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706852 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706852 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706857 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706857 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706858 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706858 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706858 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706859 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706859 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706859 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706864 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706864 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706865 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706865 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706865 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706865 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706866 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706866 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706881 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706881 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 15
   [junit4]   2> 4706882 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706882 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706882 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706883 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706883 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706883 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706888 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706888 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706889 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706889 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706889 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706890 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706890 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706890 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706895 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706895 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706896 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706896 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706896 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706897 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706897 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706897 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706902 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706903 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706903 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706903 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706904 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706904 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706904 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706905 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706910 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706910 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 5
   [junit4]   2> 4706911 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706911 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706911 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706911 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 4706912 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[2]} 0 0
   [junit4]   2> 4706912 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[3]} 0 0
   [junit4]   2> 4706917 INFO  (searcherExecutor-32287-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 4706917 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 4
   [junit4]   2> 4706925 INFO  (TEST-TestCircuitBreaker.testResponseWithCBTiming-seed#[98E16660DECD3E21]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testResponseWithCBTiming
   [junit4]   2> 4706926 INFO  (TEST-TestCircuitBreaker.testResponseWithCBTiming-seed#[98E16660DECD3E21]) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=*:*&wt=xml&debugQuery=true} status=503 QTime=0
   [junit4]   2> 4706926 ERROR (TEST-TestCircuitBreaker.testResponseWithCBTiming-seed#[98E16660DECD3E21]) [     ] o.a.s.SolrTestCaseJ4 REQUEST FAILED: q=*:*&wt=xml&debugQuery=true:org.apache.solr.common.SolrException: Circuit Breakers tripped Memory Circuit Breaker triggered as JVM heap usage values are greater than allocated threshold.Seen JVM heap memory usage 446137392 and allocated threshold 395182080
   [junit4]   2> 
   [junit4]   2> 	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:320)
   [junit4]   2> 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:216)
   [junit4]   2> 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2638)
   [junit4]   2> 	at org.apache.solr.util.TestHarness.query(TestHarness.java:346)
   [junit4]   2> 	at org.apache.solr.util.TestHarness.query(TestHarness.java:328)
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:997)
   [junit4]   2> 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:983)
   [junit4]   2> 	at org.apache.solr.util.TestCircuitBreaker.testResponseWithCBTiming(TestCircuitBreaker.java:245)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:498)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
   [junit4]   2> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
   [junit4]   2> 	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 
   [junit4]   2> 4706927 INFO  (TEST-TestCircuitBreaker.testResponseWithCBTiming-seed#[98E16660DECD3E21]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testResponseWithCBTiming
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestCircuitBreaker -Dtests.method=testResponseWithCBTiming -Dtests.seed=98E16660DECD3E21 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=sr-Latn -Dtests.timezone=America/Swift_Current -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.01s J2 | TestCircuitBreaker.testResponseWithCBTiming <<<
   [junit4]    > Throwable #1: java.lang.RuntimeException: Exception during query
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([98E16660DECD3E21:703DCB822D252401]:0)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:1023)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:983)
   [junit4]    > 	at org.apache.solr.util.TestCircuitBreaker.testResponseWithCBTiming(TestCircuitBreaker.java:245)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: org.apache.solr.common.SolrException: Circuit Breakers tripped Memory Circuit Breaker triggered as JVM heap usage values are greater than allocated threshold.Seen JVM heap memory usage 446137392 and allocated threshold 395182080
   [junit4]    > 	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:320)
   [junit4]    > 	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:216)
   [junit4]    > 	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2638)
   [junit4]    > 	at org.apache.solr.util.TestHarness.query(TestHarness.java:346)
   [junit4]    > 	at org.apache.solr.util.TestHarness.query(TestHarness.java:328)
   [junit4]    > 	at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:997)
   [junit4]    > 	... 45 more
   [junit4]   2> 4706935 INFO  (TEST-TestCircuitBreaker.testCBFakeMemoryPressure-seed#[98E16660DECD3E21]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testCBFakeMemoryPressure
   [junit4]   2> 4706935 INFO  (TEST-TestCircuitBreaker.testCBFakeMemoryPressure-seed#[98E16660DECD3E21]) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
   [junit4]   2> 4706936 INFO  (TEST-TestCircuitBreaker.testCBFakeMemoryPressure-seed#[98E16660DECD3E21]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testCBFakeMemoryPressure
   [junit4]   2> 4706943 INFO  (TEST-TestCircuitBreaker.testFakeCPUCircuitBreaker-seed#[98E16660DECD3E21]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testFakeCPUCircuitBreaker
   [junit4]   2> 4706944 INFO  (TestCircuitBreaker-32291-thread-1) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
   [junit4]   2> 4706944 INFO  (TestCircuitBreaker-32291-thread-2) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
   [junit4]   2> 4706948 INFO  (TestCircuitBreaker-32291-thread-2) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
   [junit4]   2> 4706948 INFO  (TestCircuitBreaker-32291-thread-3) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
   [junit4]   2> 4706948 INFO  (TestCircuitBreaker-32291-thread-1) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
   [junit4]   2> 4706949 INFO  (TEST-TestCircuitBreaker.testFakeCPUCircuitBreaker-seed#[98E16660DECD3E21]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testFakeCPUCircuitBreaker
   [junit4]   2> 4706956 INFO  (TEST-TestCircuitBreaker.testCBAlwaysTrips-seed#[98E16660DECD3E21]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testCBAlwaysTrips
   [junit4]   2> 4706956 INFO  (TEST-TestCircuitBreaker.testCBAlwaysTrips-seed#[98E16660DECD3E21]) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
   [junit4]   2> 4706957 INFO  (TEST-TestCircuitBreaker.testCBAlwaysTrips-seed#[98E16660DECD3E21]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testCBAlwaysTrips
   [junit4]   2> 4706962 INFO  (TEST-TestCircuitBreaker.testBuildingMemoryPressure-seed#[98E16660DECD3E21]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testBuildingMemoryPressure
   [junit4]   2> 4706963 INFO  (TestCircuitBreaker-32292-thread-1) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4706963 INFO  (TestCircuitBreaker-32292-thread-2) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4706964 INFO  (TestCircuitBreaker-32292-thread-2) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4706964 INFO  (TestCircuitBreaker-32292-thread-1) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} hits=0 status=0 QTime=0
   [junit4]   2> 4706964 INFO  (TestCircuitBreaker-32292-thread-3) [     ] o.a.s.c.S.Request [collection1]  webapp=null path=null params={q=name:"john+smith"&qt=&start=0&rows=20&version=2.2} status=503 QTime=0
   [junit4]   2> 4706965 INFO  (TEST-TestCircuitBreaker.testBuildingMemoryPressure-seed#[98E16660DECD3E21]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testBuildingMemoryPressure
   [junit4]   2> 4706965 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 4706965 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1147008537
   [junit4]   2> 4706966 INFO  (coreCloseExecutor-32294-thread-1) [     ] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@15874d35
   [junit4]   2> 4706966 INFO  (coreCloseExecutor-32294-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1 tag=SolrCore@15874d35
   [junit4]   2> 4706966 INFO  (coreCloseExecutor-32294-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2e95fe3b: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1/com.codahale.metrics.MetricRegistry@4cf6b96b
   [junit4]   2> 4706977 INFO  (coreCloseExecutor-32294-thread-1) [     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 4706978 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 4706979 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@582ec0ee: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@58ff43ce
   [junit4]   2> 4706983 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
   [junit4]   2> 4706983 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@f5352df: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@4e712cf
   [junit4]   2> 4706985 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 4706985 INFO  (SUITE-TestCircuitBreaker-seed#[98E16660DECD3E21]-worker) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5fdae275: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@5b202e9d
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.util.TestCircuitBreaker_98E16660DECD3E21-001
   [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFieldsData(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=30833, maxDocsPerChunk=282, blockShift=17), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=30833, blockSize=17)), sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=sr-Latn, timezone=America/Swift_Current
   [junit4]   2> NOTE: Linux 4.15.0-112-generic amd64/Oracle Corporation 1.8.0_281 (64-bit)/cpus=4,threads=1,free=49652768,total=526909440
   [junit4]   2> NOTE: All tests run in this JVM: [TestUtils, TestSolrQueryParser, HighlighterTest, TestSolrCLIRunExample, JSONWriterTest, OverseerStatusTest, TestTrackingShardHandlerFactory, SparseHLLTest, TestFieldResource, TestComplexPhraseQParserPlugin, DirectUpdateHandlerTest, SuggestComponentTest, DocValuesMissingTest, ImplicitSnitchTest, TestApiFramework, DeleteInactiveReplicaTest, TestRequestStatusCollectionAPI, RandomizedTaggerTest, TestSimPolicyCloud, TestCustomDocTransformer, TestBadConfig, TestBooleanSimilarityFactory, DebugComponentTest, HdfsTlogReplayBufferedWhileIndexingTest, TestDistributedStatsComponentCardinality, TestReRankQParserPlugin, TestUseDocValuesAsStored2, RemoteQueryErrorTest, QueryElevationComponentTest, DistributedQueryElevationComponentTest, TestIBSimilarityFactory, ClusterStateUpdateTest, FieldMutatingUpdateProcessorTest, TestFuzzyAnalyzedSuggestions, SolrShardReporterTest, TestSchemalessBufferedUpdates, IndexSizeTriggerMixedBoundsTest, TestNumericRangeQuery32, TestSolrDeletionPolicy1, TestPartialUpdateDeduplication, BadCopyFieldTest, TestMissingGroups, TestExplainDocTransformer, TestSimpleQParserPlugin, ConvertedLegacyTest, V2ApiIntegrationTest, TestManagedResource, ActionThrottleTest, TestSolrCloudWithHadoopAuthPlugin, BlockDirectoryTest, CleanupOldIndexTest, BadComponentTest, TestMinimalConfig, TestCursorMarkWithoutUniqueKey, SolrCloudExampleTest, TestTlogReplayVsRecovery, TestElisionMultitermQuery, LukeRequestHandlerTest, ScheduledTriggerIntegrationTest, UUIDUpdateProcessorFallbackTest, TestBlockCollapse, HttpSolrCallGetCoreTest, TestWordDelimiterFilterFactory, TestQueryUtils, TestSolrConfigHandlerConcurrent, NotRequiredUniqueKeyTest, TestSurroundQueryParser, TestRetrieveFieldsOptimizer, PropertiesRequestHandlerTest, TestSQLHandler, TestJsonRequest, AddSchemaFieldsUpdateProcessorFactoryTest, SolrMetricManagerTest, TestRequestForwarding, SuggesterTSTTest, SolrIndexSearcherTest, RegexBytesRefFilterTest, TestNonDefinedSimilarityFactory, TestReplicationHandlerBackup, BasicDistributedZk2Test, TestCloudManagedSchema, PKIAuthenticationIntegrationTest, ShardSplitTest, SOLR749Test, TestFieldCacheSort, TestOmitPositions, CachingDirectoryFactoryTest, TestMaxTokenLenTokenizer, PeerSyncReplicationTest, NestedAtomicUpdateTest, TestAnalyzedSuggestions, TlogReplayBufferedWhileIndexingTest, ExternalFileFieldSortTest, SSLMigrationTest, TestCollectionsAPIViaSolrCloudCluster, NodeLostTriggerTest, PeerSyncWithBufferUpdatesTest, TriggerIntegrationTest, TestCloudRecovery, DirectoryFactoryTest, AlternateDirectoryTest, BigEndianAscendingWordDeserializerTest, TestFilteredDocIdSet, TestTolerantSearch, AuthToolTest, TestRandomDVFaceting, TestQueryingOnDownCollection, DeleteShardTest, IndexBasedSpellCheckerTest, TestReloadAndDeleteDocs, BackupRestoreApiErrorConditionsTest, CdcrOpsAndBoundariesTest, RootFieldTest, SpatialRPTFieldTypeTest, TestPayloadCheckQParserPlugin, MoveReplicaHDFSTest, MoveReplicaTest, NestedShardedAtomicUpdateTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerRolesTest, RestartWhileUpdatingTest, ShardRoutingCustomTest, ShardRoutingTest, SyncSliceTest, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestDistributedMap, TestDownShardTolerantSearch, TestDynamicFieldNamesIndexCorrectly, TestLeaderElectionZkExpiry, TestPrepRecovery, TestSkipOverseerOperations, TestStressCloudBlindAtomicUpdates, TestZkChroot, ZkCLITest, ZkFailoverTest, ZkSolrClientTest, ConcurrentCreateCollectionTest, HdfsCloudIncrementalBackupTest, LocalFSCloudIncrementalBackupTest, PurgeGraphTest, TestCollectionAPI, ExecutePlanActionTest, MetricTriggerTest, NodeAddedTriggerIntegrationTest, SearchRateTriggerIntegrationTest, TestPolicyCloud, TestSimExecutePlanAction, TestSimExtremeIndexing, TestSimGenericDistributedQueue, TestSimNodeAddedTrigger, TestSimNodeLostTrigger, TestSnapshotCloudManager, BaseCdcrDistributedZkTest, CdcrRequestHandlerTest, CdcrVersionReplicationTest, CdcrWithNodesRestartsTest, FacetPivotSmallTest, InfixSuggestersTest, MoreLikeThisComponentTest, PhrasesIdentificationComponentTest, ShardsWhitelistTest, SuggestComponentContextFilterQueryTest, XmlInterpolationTest, FastVectorHighlighterTest, HighlighterConfigTest, HighlighterMaxOffsetTest, TestUnifiedSolrHighlighter, TestLegacyField, TestLegacyNumericUtils, TestNumericTokenStream, JvmMetricsTest, SolrJmxReporterTest, TestRemoteStreaming, TestStreamBody, TestUnInvertedFieldException, TestWriterPerf, TestGraphMLResponseWriter, TestJavabinTupleStreamParser, TestPHPSerializedResponseWriter, TestPushWriter, TestRawResponseWriter, TestRawTransformer, TestChildDocTransformer, TestChildDocTransformerHierarchy, TestRestManager, TestBulkSchemaAPI, DateFieldTest, DateRangeFieldTest, EnumFieldTest, IndexSchemaTest, ManagedSchemaRoundRobinCloudTest, OpenExchangeRatesOrgProviderTest, PolyFieldTest, PreAnalyzedFieldManagedSchemaCloudTest, SchemaVersionSpecificBehaviorTest, TestManagedSchemaAPI, TestManagedSchemaThreadSafety, TestTextField, TestUseDocValuesAsStored, CurrencyRangeFacetCloudTest, LargeFieldTest, RankQParserPluginTest, RankQueryTest, ReturnFieldsTest, SignificantTermsQParserPluginTest, TestCollapseQParserPlugin, TestDocSet, TestGraphTermsQParserPlugin, TestHashQParserPlugin, TestInitQParser, TestLFUCache, TestJsonFacetErrors, TestJsonFacetRefinement, TestJsonFacets, TestUninvertingReader, AddBlockUpdateTest, AnalysisErrorHandlingTest, CdcrUpdateLogTest, SolrIndexSplitterTest, TestInPlaceUpdateWithRouteField, TestIndexingPerformance, TestUpdate, TransactionLogTest, UpdateLogTest, ClassificationUpdateProcessorFactoryTest, ClassificationUpdateProcessorIntegrationTest, DefaultValueUpdateProcessorTest, DocExpirationUpdateProcessorFactoryTest, IgnoreLargeDocumentProcessorFactoryTest, PreAnalyzedUpdateProcessorTest, RegexBoostProcessorTest, ScriptEngineTest, StatelessScriptUpdateProcessorFactoryTest, UpdateRequestProcessorFactoryTest, XMLAtomicUpdateMultivalueTest, SolrLogPostToolTest, TestCircuitBreaker]
   [junit4] Completed [919/934 (1!)] on J2 in 1.01s, 5 tests, 1 error <<< FAILURES!

[...truncated 50856 lines...]

[JENKINS] Lucene » Lucene-Solr-NightlyTests-8.x - Build # 205 - Still Unstable!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-NightlyTests-8.x/205/

1 tests failed.
FAILED:  org.apache.solr.handler.component.TestDistributedStatsComponentCardinality.test

Error Message:
Captured an uncaught exception in thread: Thread[id=16945, name=Thread-5727, state=RUNNABLE, group=TGRP-TestDistributedStatsComponentCardinality]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16945, name=Thread-5727, state=RUNNABLE, group=TGRP-TestDistributedStatsComponentCardinality]
	at __randomizedtesting.SeedInfo.seed([CFEE770EA3E8BBF7:47BA48D40D14D60F]:0)
Caused by: junit.framework.AssertionFailedError: .responseHeader.status:200!=0
	at __randomizedtesting.SeedInfo.seed([CFEE770EA3E8BBF7]:0)
	at junit.framework.Assert.fail(Assert.java:57)
	at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:1006)
	at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:1033)
	at org.apache.solr.BaseDistributedSearchTestCase$5.run(BaseDistributedSearchTestCase.java:694)




Build Log:
[...truncated 13691 lines...]
   [junit4] Suite: org.apache.solr.handler.component.TestDistributedStatsComponentCardinality
   [junit4]   2> 745227 INFO  (SUITE-TestDistributedStatsComponentCardinality-seed#[CFEE770EA3E8BBF7]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 745227 INFO  (SUITE-TestDistributedStatsComponentCardinality-seed#[CFEE770EA3E8BBF7]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 745228 INFO  (SUITE-TestDistributedStatsComponentCardinality-seed#[CFEE770EA3E8BBF7]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/data-dir-58-001
   [junit4]   2> 745228 INFO  (SUITE-TestDistributedStatsComponentCardinality-seed#[CFEE770EA3E8BBF7]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 745230 INFO  (SUITE-TestDistributedStatsComponentCardinality-seed#[CFEE770EA3E8BBF7]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-9062)
   [junit4]   2> 745230 INFO  (SUITE-TestDistributedStatsComponentCardinality-seed#[CFEE770EA3E8BBF7]-worker) [     ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
   [junit4]   2> 745340 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/control/cores/collection1
   [junit4]   2> 745342 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 745342 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 745342 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 745348 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 745348 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 745348 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 745350 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6b799d9a{/,null,AVAILABLE}
   [junit4]   2> 745350 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@75d30add{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:42481}
   [junit4]   2> 745350 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server Started @745386ms
   [junit4]   2> 745350 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=42481, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/control/cores}
   [junit4]   2> 745351 ERROR (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 745351 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 745351 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 745351 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 745351 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 745351 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-09T14:47:07.183Z
   [junit4]   2> 745351 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/control/solr.xml
   [junit4]   2> 745353 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 745353 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 745355 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 745459 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 745460 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@368499eb[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 745460 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@368499eb[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 745463 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@7ebfe4cf[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 745463 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@7ebfe4cf[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 745464 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 745476 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 745476 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 745493 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 745500 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 745501 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 745502 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/control/cores
   [junit4]   2> 745502 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 745510 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.c.SolrResourceLoader Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 745526 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 745549 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 745556 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 745563 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 745563 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 745563 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/control/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/control/cores/collection1/data/]
   [junit4]   2> 745565 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=56.669921875, floorSegmentMB=1.8837890625, forceMergeDeletesPctAllowed=1.5454542240891744, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0, deletesPctAllowed=24.384834630446584
   [junit4]   2> 745622 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 745622 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 745623 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 745623 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 745624 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=86.1728515625, floorSegmentMB=1.033203125, forceMergeDeletesPctAllowed=17.489774720136385, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=47.41778768882654
   [junit4]   2> 745624 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 745637 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 745713 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 745713 INFO  (coreLoadExecutor-7302-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693766099974225920
   [junit4]   2> 745714 INFO  (searcherExecutor-7304-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 745714 INFO  (searcherExecutor-7304-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 745714 INFO  (searcherExecutor-7304-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 745714 INFO  (searcherExecutor-7304-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 745715 INFO  (searcherExecutor-7304-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 745715 INFO  (searcherExecutor-7304-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 745715 INFO  (searcherExecutor-7304-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 745715 INFO  (searcherExecutor-7304-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 745715 INFO  (searcherExecutor-7304-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 745717 INFO  (searcherExecutor-7304-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 745834 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard0/cores/collection1
   [junit4]   2> 745834 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 745834 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 745834 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 745835 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 745835 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 745835 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 745835 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@47859c8e{/,null,AVAILABLE}
   [junit4]   2> 745836 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@278de5ea{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:36195}
   [junit4]   2> 745836 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server Started @745872ms
   [junit4]   2> 745836 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=36195, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard0/cores}
   [junit4]   2> 745836 ERROR (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 745836 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 745836 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 745836 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 745836 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 745836 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-09T14:47:07.668Z
   [junit4]   2> 745836 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard0/solr.xml
   [junit4]   2> 745839 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 745839 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 745840 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 746967 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 746968 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@4a0ef728[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 746968 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@4a0ef728[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 746971 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@24c5534b[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 746971 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@24c5534b[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 746972 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 746984 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 746984 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 747004 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 747015 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 747016 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 747017 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard0/cores
   [junit4]   2> 747017 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 747025 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.c.SolrResourceLoader Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 747544 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 747568 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 747576 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 747583 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 747584 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 747584 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard0/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard0/cores/collection1/data/]
   [junit4]   2> 747586 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=56.669921875, floorSegmentMB=1.8837890625, forceMergeDeletesPctAllowed=1.5454542240891744, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0, deletesPctAllowed=24.384834630446584
   [junit4]   2> 747640 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 747640 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 747641 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 747641 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 747642 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=86.1728515625, floorSegmentMB=1.033203125, forceMergeDeletesPctAllowed=17.489774720136385, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=47.41778768882654
   [junit4]   2> 747643 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 747658 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 747720 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 747720 INFO  (coreLoadExecutor-7321-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693766102078717952
   [junit4]   2> 747722 INFO  (searcherExecutor-7323-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 747722 INFO  (searcherExecutor-7323-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 747722 INFO  (searcherExecutor-7323-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 747722 INFO  (searcherExecutor-7323-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 747722 INFO  (searcherExecutor-7323-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 747722 INFO  (searcherExecutor-7323-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 747722 INFO  (searcherExecutor-7323-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 747723 INFO  (searcherExecutor-7323-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 747723 INFO  (searcherExecutor-7323-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 747724 INFO  (searcherExecutor-7323-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 748327 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard1/cores/collection1
   [junit4]   2> 748328 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 748328 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 748328 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 748329 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 748329 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 748329 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 748329 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@72885c26{/,null,AVAILABLE}
   [junit4]   2> 748330 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@366bf418{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:33455}
   [junit4]   2> 748330 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server Started @748365ms
   [junit4]   2> 748330 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=33455, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard1/cores}
   [junit4]   2> 748330 ERROR (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 748330 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 748330 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 748330 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 748330 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 748330 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-09T14:47:10.162Z
   [junit4]   2> 748330 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard1/solr.xml
   [junit4]   2> 748333 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 748333 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 748334 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 748602 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 748603 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3bb462c7[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 748603 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3bb462c7[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 748606 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@4ecb414a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 748606 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@4ecb414a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 748607 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 748617 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 748617 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 748645 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 748656 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 748656 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 748658 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard1/cores
   [junit4]   2> 748658 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 748666 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.c.SolrResourceLoader Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 748682 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 748707 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 748719 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 748728 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 748729 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 748729 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard1/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard1/cores/collection1/data/]
   [junit4]   2> 748731 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=56.669921875, floorSegmentMB=1.8837890625, forceMergeDeletesPctAllowed=1.5454542240891744, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0, deletesPctAllowed=24.384834630446584
   [junit4]   2> 748785 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 748785 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 748787 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 748787 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 748788 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=86.1728515625, floorSegmentMB=1.033203125, forceMergeDeletesPctAllowed=17.489774720136385, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=47.41778768882654
   [junit4]   2> 748789 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 748804 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 748875 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 748875 INFO  (coreLoadExecutor-7340-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693766103289823232
   [junit4]   2> 748887 INFO  (searcherExecutor-7342-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 748888 INFO  (searcherExecutor-7342-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 748888 INFO  (searcherExecutor-7342-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 748888 INFO  (searcherExecutor-7342-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 748888 INFO  (searcherExecutor-7342-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 748888 INFO  (searcherExecutor-7342-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 748888 INFO  (searcherExecutor-7342-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 748888 INFO  (searcherExecutor-7342-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 748888 INFO  (searcherExecutor-7342-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 748890 INFO  (searcherExecutor-7342-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 749485 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard2/cores/collection1
   [junit4]   2> 749486 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 749486 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 749486 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 749489 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 749489 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 749489 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 749491 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@34fdc8b8{/,null,AVAILABLE}
   [junit4]   2> 749491 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@14f49439{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:38721}
   [junit4]   2> 749492 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server Started @749527ms
   [junit4]   2> 749492 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=38721, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard2/cores}
   [junit4]   2> 749492 ERROR (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 749492 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 749492 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 749492 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 749492 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 749492 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-09T14:47:11.324Z
   [junit4]   2> 749492 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard2/solr.xml
   [junit4]   2> 749494 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 749494 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 749496 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 749763 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 749764 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@48c44fa8[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 749764 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@48c44fa8[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 749773 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3a1d4174[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 749773 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3a1d4174[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 749774 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 749784 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 749784 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 749803 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 749813 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 749813 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 749815 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard2/cores
   [junit4]   2> 749815 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 749823 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.c.SolrResourceLoader Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 749839 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 749875 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 749881 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 749888 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 749888 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 749889 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard2/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard2/cores/collection1/data/]
   [junit4]   2> 749890 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=56.669921875, floorSegmentMB=1.8837890625, forceMergeDeletesPctAllowed=1.5454542240891744, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0, deletesPctAllowed=24.384834630446584
   [junit4]   2> 749941 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 749941 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 749942 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 749942 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 749943 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=86.1728515625, floorSegmentMB=1.033203125, forceMergeDeletesPctAllowed=17.489774720136385, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=47.41778768882654
   [junit4]   2> 749944 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 749959 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 750018 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 750018 INFO  (coreLoadExecutor-7359-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693766104488345600
   [junit4]   2> 750020 INFO  (searcherExecutor-7361-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 750020 INFO  (searcherExecutor-7361-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 750020 INFO  (searcherExecutor-7361-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 750020 INFO  (searcherExecutor-7361-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 750020 INFO  (searcherExecutor-7361-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 750020 INFO  (searcherExecutor-7361-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 750021 INFO  (searcherExecutor-7361-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 750021 INFO  (searcherExecutor-7361-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 750021 INFO  (searcherExecutor-7361-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 750023 INFO  (searcherExecutor-7361-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 750629 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard3/cores/collection1
   [junit4]   2> 750630 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 750630 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 750630 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 750636 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 750637 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 750637 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 750637 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6cd0a4fc{/,null,AVAILABLE}
   [junit4]   2> 750637 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@36d3dd47{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:39631}
   [junit4]   2> 750637 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server Started @750673ms
   [junit4]   2> 750637 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=39631, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard3/cores}
   [junit4]   2> 750637 ERROR (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 750637 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 750637 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 750637 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 750637 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 750638 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-09T14:47:12.470Z
   [junit4]   2> 750638 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard3/solr.xml
   [junit4]   2> 750640 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 750640 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 750641 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 751596 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 751597 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@7c0e5a9[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 751597 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@7c0e5a9[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 751600 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@700c257[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 751601 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@700c257[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 751602 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 751612 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 751612 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 751631 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 751654 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 751654 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 751656 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard3/cores
   [junit4]   2> 751656 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 751682 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.c.SolrResourceLoader Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 751712 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 751737 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 751743 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 751750 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 751750 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 751750 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard3/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard3/cores/collection1/data/]
   [junit4]   2> 751753 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=56.669921875, floorSegmentMB=1.8837890625, forceMergeDeletesPctAllowed=1.5454542240891744, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0, deletesPctAllowed=24.384834630446584
   [junit4]   2> 751808 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 751808 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 751809 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 751809 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 751810 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=86.1728515625, floorSegmentMB=1.033203125, forceMergeDeletesPctAllowed=17.489774720136385, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=47.41778768882654
   [junit4]   2> 751811 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 751826 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 751887 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 751887 INFO  (coreLoadExecutor-7378-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693766106448134144
   [junit4]   2> 751889 INFO  (searcherExecutor-7380-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 751889 INFO  (searcherExecutor-7380-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 751889 INFO  (searcherExecutor-7380-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 751889 INFO  (searcherExecutor-7380-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 751889 INFO  (searcherExecutor-7380-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 751889 INFO  (searcherExecutor-7380-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 751889 INFO  (searcherExecutor-7380-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 751889 INFO  (searcherExecutor-7380-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 751890 INFO  (searcherExecutor-7380-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
   [junit4]   2> 751902 INFO  (searcherExecutor-7380-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 752498 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard4/cores/collection1
   [junit4]   2> 752499 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 752499 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 752499 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 752500 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 752500 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 752500 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 752500 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@556a49d6{/,null,AVAILABLE}
   [junit4]   2> 752501 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@128e52a8{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:34563}
   [junit4]   2> 752501 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.s.Server Started @752537ms
   [junit4]   2> 752501 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, hostPort=34563, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard4/cores}
   [junit4]   2> 752501 ERROR (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 752501 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 752501 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 752501 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 752501 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 752501 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-09T14:47:14.333Z
   [junit4]   2> 752501 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard4/solr.xml
   [junit4]   2> 752503 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 752504 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 752505 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 752876 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 752877 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@4fd6a1f[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 752877 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@4fd6a1f[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 752880 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@548fd230[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 752880 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@548fd230[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 752881 WARN  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 752892 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 752892 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 752913 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 752923 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 752923 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 752925 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard4/cores
   [junit4]   2> 752925 INFO  (TEST-TestDistributedStatsComponentCardinality.test-seed#[CFEE770EA3E8BBF7]) [     ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 752933 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.c.SolrResourceLoader Added 2 libs to classloader, from paths: [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib, /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/lib/classes]
   [junit4]   2> 752949 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 752988 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 752996 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
   [junit4]   2> 753006 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from configset /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1, trusted=true
   [junit4]   2> 753006 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53876b1f
   [junit4]   2> 753006 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard4/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.handler.component.TestDistributedStatsComponentCardinality_CFEE770EA3E8BBF7-001/tempDir-001/shard4/cores/collection1/data/]
   [junit4]   2> 753008 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=7, maxMergeAtOnceExplicit=2, maxMergedSegmentMB=56.669921875, floorSegmentMB=1.8837890625, forceMergeDeletesPctAllowed=1.5454542240891744, segmentsPerTier=43.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0, deletesPctAllowed=24.384834630446584
   [junit4]   2> 753064 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 753064 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 753066 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 753066 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 753066 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=86.1728515625, floorSegmentMB=1.033203125, forceMergeDeletesPctAllowed=17.489774720136385, segmentsPerTier=48.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0, deletesPctAllowed=47.41778768882654
   [junit4]   2> 753067 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
   [junit4]   2> 753082 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
   [junit4]   2> 753149 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 753149 INFO  (coreLoadExecutor-7397-thread-1) [    x:collection1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693766107771437056
   [junit4]   2> 753152 INFO  (searcherExecutor-7399-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
   [junit4]   2> 753153 INFO  (searcherExecutor-7399-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
   [junit4]   2> 753153 INFO  (searcherExecutor-7399-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
   [junit4]   2> 753153 INFO  (searcherExecutor-7399-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
   [junit4]   2> 753154 INFO  (searcherExecutor-7399-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
   [junit4]   2> 753154 INFO  (searcherExecutor-7399-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
   [junit4]   2> 753155 INFO  (searcherExecutor-7399-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
   [junit4]   2> 753155 INFO  (searcherExecutor-7399-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
   [junit4]   2> 753155 INFO  (searcherExecutor-7399-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: pe

[...truncated too long message...]

eporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5d91a48c: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@3cbe526b
   [junit4]   2> 1010115 INFO  (closeThreadPool-7449-thread-12) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 1010115 INFO  (closeThreadPool-7449-thread-12) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7b2c0867: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@3cbe526b
   [junit4]   2> 1010151 INFO  (closeThreadPool-7449-thread-2) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@75d30add{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 1010152 INFO  (closeThreadPool-7449-thread-2) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6b799d9a{/,null,STOPPED}
   [junit4]   2> 1010152 INFO  (closeThreadPool-7449-thread-2) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1010166 INFO  (closeThreadPool-7449-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 1010166 INFO  (closeThreadPool-7449-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1c1e7d48: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@38afaf7b
   [junit4]   2> 1010172 INFO  (closeThreadPool-7449-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
   [junit4]   2> 1010172 INFO  (closeThreadPool-7449-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@11edd550: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@36bb974f
   [junit4]   2> 1010173 INFO  (closeThreadPool-7449-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 1010173 INFO  (closeThreadPool-7449-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@f066f4a: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@3cbe526b
   [junit4]   2> 1010176 INFO  (closeThreadPool-7449-thread-6) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@128e52a8{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 1010176 INFO  (closeThreadPool-7449-thread-6) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@556a49d6{/,null,STOPPED}
   [junit4]   2> 1010176 INFO  (closeThreadPool-7449-thread-6) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1010185 INFO  (closeThreadPool-7449-thread-12) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@36d3dd47{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 1010185 INFO  (closeThreadPool-7449-thread-12) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6cd0a4fc{/,null,STOPPED}
   [junit4]   2> 1010185 INFO  (closeThreadPool-7449-thread-12) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1010189 INFO  (closeThreadPool-7449-thread-5) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@608f9742{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 1010189 INFO  (closeThreadPool-7449-thread-5) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@443968f0{/,null,STOPPED}
   [junit4]   2> 1010189 INFO  (closeThreadPool-7449-thread-5) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1010190 INFO  (closeThreadPool-7449-thread-4) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@358ed747{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 1010190 INFO  (closeThreadPool-7449-thread-4) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4758b695{/,null,STOPPED}
   [junit4]   2> 1010190 INFO  (closeThreadPool-7449-thread-4) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1010190 INFO  (closeThreadPool-7449-thread-10) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@366bf418{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 1010190 INFO  (closeThreadPool-7449-thread-10) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@72885c26{/,null,STOPPED}
   [junit4]   2> 1010190 INFO  (closeThreadPool-7449-thread-10) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1010194 INFO  (closeThreadPool-7449-thread-11) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@14f49439{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 1010194 INFO  (closeThreadPool-7449-thread-11) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@34fdc8b8{/,null,STOPPED}
   [junit4]   2> 1010194 INFO  (closeThreadPool-7449-thread-11) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 1010197 INFO  (closeThreadPool-7449-thread-1) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@278de5ea{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 1010197 INFO  (closeThreadPool-7449-thread-1) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@47859c8e{/,null,STOPPED}
   [junit4]   2> 1010197 INFO  (closeThreadPool-7449-thread-1) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDistributedStatsComponentCardinality -Dtests.method=test -Dtests.seed=CFEE770EA3E8BBF7 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=ar-SY -Dtests.timezone=Europe/Malta -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR    265s J2 | TestDistributedStatsComponentCardinality.test <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=16945, name=Thread-5727, state=RUNNABLE, group=TGRP-TestDistributedStatsComponentCardinality]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([CFEE770EA3E8BBF7:47BA48D40D14D60F]:0)
   [junit4]    > Caused by: junit.framework.AssertionFailedError: .responseHeader.status:200!=0
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([CFEE770EA3E8BBF7]:0)
   [junit4]    > 	at junit.framework.Assert.fail(Assert.java:57)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.compareSolrResponses(BaseDistributedSearchTestCase.java:1006)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.compareResponses(BaseDistributedSearchTestCase.java:1033)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$5.run(BaseDistributedSearchTestCase.java:694)
   [junit4]   2> 1010206 INFO  (SUITE-TestDistributedStatsComponentCardinality-seed#[CFEE770EA3E8BBF7]-worker) [     ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene87): {multiDefault=PostingsFormat(name=LuceneFixedGap), _root_=PostingsFormat(name=LuceneVarGapDocFreqInterval), string_s=Lucene84, id=PostingsFormat(name=MockRandom)}, docValues:{int_i=DocValuesFormat(name=Lucene80), range_facet_l_dv=DocValuesFormat(name=Asserting), intDefault=DocValuesFormat(name=Lucene80), int_i_prehashed_l=DocValuesFormat(name=Lucene80), range_facet_l=DocValuesFormat(name=Lucene80), long_l_prehashed_l=DocValuesFormat(name=Lucene80), long_l=DocValuesFormat(name=Asserting), _version_=DocValuesFormat(name=Lucene80), string_s_prehashed_l=DocValuesFormat(name=Asserting), id_i1=DocValuesFormat(name=Direct), range_facet_i_dv=DocValuesFormat(name=Lucene80), intDvoDefault=DocValuesFormat(name=Lucene80), timestamp=DocValuesFormat(name=Lucene80)}, maxPointsInLeafNode=1760, maxMBSortInHeap=7.140497195367099, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=ar-SY, timezone=Europe/Malta
   [junit4]   2> NOTE: Linux 4.15.0-112-generic amd64/Oracle Corporation 1.8.0_281 (64-bit)/cpus=4,threads=1,free=66653888,total=532676608
   [junit4]   2> NOTE: All tests run in this JVM: [BigEndianAscendingWordDeserializerTest, TestExplainDocTransformer, ScheduledTriggerIntegrationTest, AddBlockUpdateTest, SpatialHeatmapFacetsTest, TestCryptoKeys, SolrShardReporterTest, TestSizeLimitedDistributedMap, TestBlockCollapse, TestOnReconnectListenerSupport, TestSolrCLIRunExample, TestFuzzyAnalyzedSuggestions, OverseerModifyCollectionTest, TestFieldCacheSort, LeaderVoteWaitTimeoutTest, ZkCLITest, SolrIndexSearcherTest, JvmMetricsTest, TestSimpleQParserPlugin, CheckHdfsIndexTest, ThreadDumpHandlerTest, TestXIncludeConfig, AuthToolTest, ClassificationUpdateProcessorFactoryTest, SuggestComponentTest, TestFieldResource, ChaosMonkeyNothingIsSafeWithPullReplicasTest, AddSchemaFieldsUpdateProcessorFactoryTest, ComputePlanActionTest, TestBooleanSimilarityFactory, TestJsonRequest, HighlighterTest, XmlInterpolationTest, MoveReplicaTest, TestSolrCachePerf, TestSolrDeletionPolicy1, RootFieldTest, SolrMetricManagerTest, TestSurroundQueryParser, TestCoreDiscovery, TestCharFilters, TestReloadAndDeleteDocs, TestCursorMarkWithoutUniqueKey, TestCloudInspectUtil, HttpPartitionTest, ShardTermsTest, TestOmitPositions, TestUtilizeNode, TestPKIAuthenticationPlugin, TestApiFramework, TestDistributedStatsComponentCardinality]
   [junit4] Completed [191/934 (1!)] on J2 in 265.87s, 1 test, 1 error <<< FAILURES!

[...truncated 53332 lines...]

[JENKINS] Lucene » Lucene-Solr-NightlyTests-8.x - Build # 204 - Still unstable!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-NightlyTests-8.x/204/

4 tests failed.
FAILED:  org.apache.lucene.index.TestDemoParallelLeafReader.testRandomMultipleSchemaGens

Error Message:
refusing to delete any files: this IndexWriter hit an unrecoverable exception

Stack Trace:
org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
	at org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:349)
	at org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:669)
	at org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:589)
	at org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:531)
	at org.apache.lucene.index.IndexWriter.checkpoint(IndexWriter.java:2716)
	at org.apache.lucene.index.IndexWriter.publishFlushedSegment(IndexWriter.java:2794)
	at org.apache.lucene.index.IndexWriter.lambda$publishFlushedSegments$22(IndexWriter.java:5384)
	at org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:119)
	at org.apache.lucene.index.DocumentsWriterFlushQueue.tryPurge(DocumentsWriterFlushQueue.java:150)
	at org.apache.lucene.index.DocumentsWriter.purgeFlushTickets(DocumentsWriter.java:191)
	at org.apache.lucene.index.IndexWriter.publishFlushedSegments(IndexWriter.java:5364)
	at org.apache.lucene.index.IndexWriter.access$300(IndexWriter.java:219)
	at org.apache.lucene.index.IndexWriter$1.afterSegmentsFlushed(IndexWriter.java:446)
	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:521)
	at org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:390)
	at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:436)
	at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1471)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1757)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1400)
	at org.apache.lucene.index.TestDemoParallelLeafReader.testRandomMultipleSchemaGens(TestDemoParallelLeafReader.java:981)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjg4y_48/_0_Lucene80_0.dvm: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
	at java.nio.file.Files.newOutputStream(Files.java:216)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:672)
	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesConsumer.<init>(Lucene80DocValuesConsumer.java:82)
	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesFormat.fieldsConsumer(Lucene80DocValuesFormat.java:161)
	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:227)
	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:163)
	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:109)
	at org.apache.lucene.index.NumericDocValuesWriter.flush(NumericDocValuesWriter.java:108)
	at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:345)
	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:225)
	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:476)
	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:656)
	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3911)
	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3886)
	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2089)
	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2069)
	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:747)
	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)


FAILED:  org.apache.lucene.index.TestDemoParallelLeafReader.testRandomMultipleSchemaGensSameField

Error Message:
Captured an uncaught exception in thread: Thread[id=21438, name=Lucene Merge Thread #21, state=RUNNABLE, group=TGRP-TestDemoParallelLeafReader]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=21438, name=Lucene Merge Thread #21, state=RUNNABLE, group=TGRP-TestDemoParallelLeafReader]
Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjefo_47/_0_Lucene85FieldsIndexfile_pointers_1.tmp: Too many open files
	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjefo_47/_0_Lucene85FieldsIndexfile_pointers_1.tmp: Too many open files
	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
	at java.nio.file.Files.newOutputStream(Files.java:216)
	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
	at org.apache.lucene.store.FSDirectory.createTempOutput(FSDirectory.java:267)
	at org.apache.lucene.store.MockDirectoryWrapper.createTempOutput(MockDirectoryWrapper.java:707)
	at org.apache.lucene.store.FilterDirectory.createTempOutput(FilterDirectory.java:79)
	at org.apache.lucene.store.TrackingDirectoryWrapper.createTempOutput(TrackingDirectoryWrapper.java:51)
	at org.apache.lucene.codecs.compressing.FieldsIndexWriter.<init>(FieldsIndexWriter.java:80)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:126)
	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:130)
	at org.apache.lucene.codecs.lucene87.Lucene87StoredFieldsFormat.fieldsWriter(Lucene87StoredFieldsFormat.java:141)
	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:48)
	at org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter(StoredFieldsConsumer.java:48)
	at org.apache.lucene.index.StoredFieldsConsumer.startDocument(StoredFieldsConsumer.java:55)
	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:449)
	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:485)
	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:208)
	at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:415)
	at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1471)
	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1757)
	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1400)
	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:731)
	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)


FAILED:  org.apache.solr.handler.TestSnapshotCoreBackup.testBackupAfterSoftCommit

Error Message:
Failed to backup core=collection1 because org.apache.lucene.index.CorruptIndexException: codec footer mismatch (file truncated?): actual footer=808464432 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(MockIndexInputWrapper(RAMInputStream(name=_0.si))))

Stack Trace:
org.apache.solr.common.SolrException: Failed to backup core=collection1 because org.apache.lucene.index.CorruptIndexException: codec footer mismatch (file truncated?): actual footer=808464432 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(MockIndexInputWrapper(RAMInputStream(name=_0.si))))
	at __randomizedtesting.SeedInfo.seed([B03222CD88624505:CBFA6C29598F1B99]:0)
	at org.apache.solr.handler.admin.BackupCoreOp.execute(BackupCoreOp.java:89)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.TestSnapshotCoreBackup.testBackupAfterSoftCommit(TestSnapshotCoreBackup.java:210)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.lucene.index.CorruptIndexException: codec footer mismatch (file truncated?): actual footer=808464432 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(MockIndexInputWrapper(RAMInputStream(name=_0.si))))
	at org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:523)
	at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:414)
	at org.apache.solr.core.backup.repository.BackupRepository.copyIndexFileFrom(BackupRepository.java:217)
	at org.apache.solr.core.backup.repository.LocalFileSystemRepository.copyIndexFileFrom(LocalFileSystemRepository.java:165)
	at org.apache.solr.core.backup.repository.BackupRepository.copyFileFrom(BackupRepository.java:179)
	at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:271)
	at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:158)
	at org.apache.solr.handler.admin.BackupCoreOp.execute(BackupCoreOp.java:86)
	... 47 more


FAILED:  org.apache.solr.handler.TestSnapshotCoreBackup.testBackupWithDocsNotSearchable

Error Message:
Failed to backup core=collection1 because org.apache.lucene.index.CorruptIndexException: codec footer mismatch (file truncated?): actual footer=808464432 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(MockIndexInputWrapper(RAMInputStream(name=_0.si))))

Stack Trace:
org.apache.solr.common.SolrException: Failed to backup core=collection1 because org.apache.lucene.index.CorruptIndexException: codec footer mismatch (file truncated?): actual footer=808464432 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(MockIndexInputWrapper(RAMInputStream(name=_0.si))))
	at __randomizedtesting.SeedInfo.seed([B03222CD88624505:4B466C181C62B147]:0)
	at org.apache.solr.handler.admin.BackupCoreOp.execute(BackupCoreOp.java:89)
	at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:367)
	at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:397)
	at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:181)
	at org.apache.solr.handler.TestSnapshotCoreBackup.testBackupWithDocsNotSearchable(TestSnapshotCoreBackup.java:68)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.lucene.index.CorruptIndexException: codec footer mismatch (file truncated?): actual footer=808464432 vs expected footer=-1071082520 (resource=BufferedChecksumIndexInput(MockIndexInputWrapper(RAMInputStream(name=_0.si))))
	at org.apache.lucene.codecs.CodecUtil.validateFooter(CodecUtil.java:523)
	at org.apache.lucene.codecs.CodecUtil.checkFooter(CodecUtil.java:414)
	at org.apache.solr.core.backup.repository.BackupRepository.copyIndexFileFrom(BackupRepository.java:217)
	at org.apache.solr.core.backup.repository.LocalFileSystemRepository.copyIndexFileFrom(LocalFileSystemRepository.java:165)
	at org.apache.solr.core.backup.repository.BackupRepository.copyFileFrom(BackupRepository.java:179)
	at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:271)
	at org.apache.solr.handler.SnapShooter.createSnapshot(SnapShooter.java:158)
	at org.apache.solr.handler.admin.BackupCoreOp.execute(BackupCoreOp.java:86)
	... 47 more




Build Log:
[...truncated 1630 lines...]
   [junit4] Suite: org.apache.lucene.index.TestDemoParallelLeafReader
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDemoParallelLeafReader -Dtests.method=testRandomMultipleSchemaGens -Dtests.seed=58D85E1FF5DF2279 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=is -Dtests.timezone=Antarctica/Davis -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   49.4s J2 | TestDemoParallelLeafReader.testRandomMultipleSchemaGens <<<
   [junit4]    > Throwable #1: org.apache.lucene.store.AlreadyClosedException: refusing to delete any files: this IndexWriter hit an unrecoverable exception
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.ensureOpen(IndexFileDeleter.java:349)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.deleteFiles(IndexFileDeleter.java:669)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.decRef(IndexFileDeleter.java:589)
   [junit4]    > 	at org.apache.lucene.index.IndexFileDeleter.checkpoint(IndexFileDeleter.java:531)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.checkpoint(IndexWriter.java:2716)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.publishFlushedSegment(IndexWriter.java:2794)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.lambda$publishFlushedSegments$22(IndexWriter.java:5384)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue.innerPurge(DocumentsWriterFlushQueue.java:119)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterFlushQueue.tryPurge(DocumentsWriterFlushQueue.java:150)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.purgeFlushTickets(DocumentsWriter.java:191)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.publishFlushedSegments(IndexWriter.java:5364)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.access$300(IndexWriter.java:219)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter$1.afterSegmentsFlushed(IndexWriter.java:446)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:521)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.postUpdate(DocumentsWriter.java:390)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:436)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1471)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1757)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1400)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader.testRandomMultipleSchemaGens(TestDemoParallelLeafReader.java:981)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjg4y_48/_0_Lucene80_0.dvm: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:672)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesConsumer.<init>(Lucene80DocValuesConsumer.java:82)
   [junit4]    > 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesFormat.fieldsConsumer(Lucene80DocValuesFormat.java:161)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:227)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:163)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:109)
   [junit4]    > 	at org.apache.lucene.index.NumericDocValuesWriter.flush(NumericDocValuesWriter.java:108)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:345)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:225)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:476)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:656)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3911)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3886)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2089)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2069)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:747)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)Throwable #2: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=21502, name=Lucene Merge Thread #30, state=RUNNABLE, group=TGRP-TestDemoParallelLeafReader]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjfup_48/_0_Lucene80_0.dvd: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjfup_48/_0_Lucene80_0.dvd: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:672)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesConsumer.<init>(Lucene80DocValuesConsumer.java:79)
   [junit4]    > 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesFormat.fieldsConsumer(Lucene80DocValuesFormat.java:161)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:227)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:163)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:109)
   [junit4]    > 	at org.apache.lucene.index.NumericDocValuesWriter.flush(NumericDocValuesWriter.java:108)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:345)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:225)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:476)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:656)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3911)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3886)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2089)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2069)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:747)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)Throwable #3: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=21474, name=Lucene Merge Thread #28, state=RUNNABLE, group=TGRP-TestDemoParallelLeafReader]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjf8a_47/_0_Lucene85FieldsIndex-doc_ids_0.tmp: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjf8a_47/_0_Lucene85FieldsIndex-doc_ids_0.tmp: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createTempOutput(FSDirectory.java:267)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createTempOutput(MockDirectoryWrapper.java:707)
   [junit4]    > 	at org.apache.lucene.store.FilterDirectory.createTempOutput(FilterDirectory.java:79)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createTempOutput(TrackingDirectoryWrapper.java:51)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.FieldsIndexWriter.<init>(FieldsIndexWriter.java:76)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:126)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:130)
   [junit4]    > 	at org.apache.lucene.codecs.lucene87.Lucene87StoredFieldsFormat.fieldsWriter(Lucene87StoredFieldsFormat.java:141)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:48)
   [junit4]    > 	at org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter(StoredFieldsConsumer.java:48)
   [junit4]    > 	at org.apache.lucene.index.StoredFieldsConsumer.startDocument(StoredFieldsConsumer.java:55)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:449)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:485)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:208)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:415)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1471)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1757)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1400)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:731)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)Throwable #4: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=21462, name=Lucene Merge Thread #26, state=RUNNABLE, group=TGRP-TestDemoParallelLeafReader]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjf2w_47/_0.kdi: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjf2w_47/_0.kdi: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:672)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.lucene86.Lucene86PointsWriter.<init>(Lucene86PointsWriter.java:85)
   [junit4]    > 	at org.apache.lucene.index.RandomCodec$1$1.<init>(RandomCodec.java:105)
   [junit4]    > 	at org.apache.lucene.index.RandomCodec$1.fieldsWriter(RandomCodec.java:105)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingPointsFormat.fieldsWriter(AssertingPointsFormat.java:58)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.writePoints(DefaultIndexingChain.java:302)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:231)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:476)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:656)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3911)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3886)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2089)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2069)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:747)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)Throwable #5: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=21456, name=Lucene Merge Thread #25, state=RUNNABLE, group=TGRP-TestDemoParallelLeafReader]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjeyo_47/_0_Asserting_0.dvm: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjeyo_47/_0_Asserting_0.dvm: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:672)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesConsumer.<init>(Lucene80DocValuesConsumer.java:82)
   [junit4]    > 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesFormat.fieldsConsumer(Lucene80DocValuesFormat.java:161)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat.fieldsConsumer(AssertingDocValuesFormat.java:55)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:227)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:163)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:109)
   [junit4]    > 	at org.apache.lucene.index.NumericDocValuesWriter.flush(NumericDocValuesWriter.java:108)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:345)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:225)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:476)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:656)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3911)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3886)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2089)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2069)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:747)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)Throwable #6: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=21453, name=Lucene Merge Thread #24, state=RUNNABLE, group=TGRP-TestDemoParallelLeafReader]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjevb_47/_0_Asserting_0.dvd: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjevb_47/_0_Asserting_0.dvd: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:672)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesConsumer.<init>(Lucene80DocValuesConsumer.java:79)
   [junit4]    > 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesFormat.fieldsConsumer(Lucene80DocValuesFormat.java:161)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat.fieldsConsumer(AssertingDocValuesFormat.java:55)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:227)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:163)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:109)
   [junit4]    > 	at org.apache.lucene.index.NumericDocValuesWriter.flush(NumericDocValuesWriter.java:108)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:345)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:225)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:476)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:656)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3911)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3886)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2089)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2069)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:747)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)Throwable #7: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=21449, name=Lucene Merge Thread #23, state=RUNNABLE, group=TGRP-TestDemoParallelLeafReader]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjery_47/_0_Lucene85FieldsIndexfile_pointers_1.tmp: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjery_47/_0_Lucene85FieldsIndexfile_pointers_1.tmp: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createTempOutput(FSDirectory.java:267)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createTempOutput(MockDirectoryWrapper.java:707)
   [junit4]    > 	at org.apache.lucene.store.FilterDirectory.createTempOutput(FilterDirectory.java:79)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createTempOutput(TrackingDirectoryWrapper.java:51)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.FieldsIndexWriter.<init>(FieldsIndexWriter.java:80)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:126)
   [junit4]    > 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:130)
   [junit4]    > 	at org.apache.lucene.codecs.lucene87.Lucene87StoredFieldsFormat.fieldsWriter(Lucene87StoredFieldsFormat.java:141)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:48)
   [junit4]    > 	at org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter(StoredFieldsConsumer.java:48)
   [junit4]    > 	at org.apache.lucene.index.StoredFieldsConsumer.startDocument(StoredFieldsConsumer.java:55)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:449)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:485)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:208)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:415)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1471)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1757)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1400)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:731)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)Throwable #8: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=21433, name=Lucene Merge Thread #20, state=RUNNABLE, group=TGRP-TestDemoParallelLeafReader]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbje6u_47/_0.kdi: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbje6u_47/_0.kdi: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:672)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.lucene86.Lucene86PointsWriter.<init>(Lucene86PointsWriter.java:85)
   [junit4]    > 	at org.apache.lucene.index.RandomCodec$1$1.<init>(RandomCodec.java:105)
   [junit4]    > 	at org.apache.lucene.index.RandomCodec$1.fieldsWriter(RandomCodec.java:105)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingPointsFormat.fieldsWriter(AssertingPointsFormat.java:58)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.writePoints(DefaultIndexingChain.java:302)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:231)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:476)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:656)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3911)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3886)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2089)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2069)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:747)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)Throwable #9: com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=21429, name=Lucene Merge Thread #19, state=RUNNABLE, group=TGRP-TestDemoParallelLeafReader]
   [junit4]    > Caused by: org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbje9m_47/_0_Asserting_0.dvm: Too many open files
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
   [junit4]    > Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbje9m_47/_0_Asserting_0.dvm: Too many open files
   [junit4]    > 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]    > 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]    > 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:672)
   [junit4]    > 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]    > 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]    > 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesConsumer.<init>(Lucene80DocValuesConsumer.java:82)
   [junit4]    > 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesFormat.fieldsConsumer(Lucene80DocValuesFormat.java:161)
   [junit4]    > 	at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat.fieldsConsumer(AssertingDocValuesFormat.java:55)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:227)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:163)
   [junit4]    > 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:109)
   [junit4]    > 	at org.apache.lucene.index.NumericDocValuesWriter.flush(NumericDocValuesWriter.java:108)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:345)
   [junit4]    > 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:225)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:476)
   [junit4]    > 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:656)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3911)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3886)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2089)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2069)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:747)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]    > 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]    > 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
   [junit4]   2> mar. 08, 2021 7:50:19 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #30,5,TGRP-TestDemoParallelLeafReader]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjfup_48/_0_Lucene80_0.dvd: Too many open files
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjfup_48/_0_Lucene80_0.dvd: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:672)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesConsumer.<init>(Lucene80DocValuesConsumer.java:79)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesFormat.fieldsConsumer(Lucene80DocValuesFormat.java:161)
   [junit4]   2> 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:227)
   [junit4]   2> 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:163)
   [junit4]   2> 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:109)
   [junit4]   2> 	at org.apache.lucene.index.NumericDocValuesWriter.flush(NumericDocValuesWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:345)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:225)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:476)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:656)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3911)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3886)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2089)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2069)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:747)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
   [junit4]   2> 
   [junit4]   2> mar. 08, 2021 7:50:19 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #16,5,TGRP-TestDemoParallelLeafReader]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbje3f_46/_0_Lucene85FieldsIndexfile_pointers_1.tmp: Too many open files
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbje3f_46/_0_Lucene85FieldsIndexfile_pointers_1.tmp: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createTempOutput(FSDirectory.java:267)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createTempOutput(MockDirectoryWrapper.java:707)
   [junit4]   2> 	at org.apache.lucene.store.FilterDirectory.createTempOutput(FilterDirectory.java:79)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createTempOutput(TrackingDirectoryWrapper.java:51)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.FieldsIndexWriter.<init>(FieldsIndexWriter.java:80)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsWriter.<init>(CompressingStoredFieldsWriter.java:126)
   [junit4]   2> 	at org.apache.lucene.codecs.compressing.CompressingStoredFieldsFormat.fieldsWriter(CompressingStoredFieldsFormat.java:130)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene87.Lucene87StoredFieldsFormat.fieldsWriter(Lucene87StoredFieldsFormat.java:141)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingStoredFieldsFormat.fieldsWriter(AssertingStoredFieldsFormat.java:48)
   [junit4]   2> 	at org.apache.lucene.index.StoredFieldsConsumer.initStoredFieldsWriter(StoredFieldsConsumer.java:48)
   [junit4]   2> 	at org.apache.lucene.index.StoredFieldsConsumer.startDocument(StoredFieldsConsumer.java:55)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.startStoredFields(DefaultIndexingChain.java:449)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.processDocument(DefaultIndexingChain.java:485)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:208)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:415)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1471)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.updateDocument(IndexWriter.java:1757)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1400)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:731)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
   [junit4]   2> 
   [junit4]   2> mar. 08, 2021 7:50:19 PM com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
   [junit4]   2> WARNING: Uncaught exception in thread: Thread[Lucene Merge Thread #27,5,TGRP-TestDemoParallelLeafReader]
   [junit4]   2> org.apache.lucene.index.MergePolicy$MergeException: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjf4r_47/_0_Asserting_0.dvd: Too many open files
   [junit4]   2> 	at __randomizedtesting.SeedInfo.seed([58D85E1FF5DF2279]:0)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:705)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:696)
   [junit4]   2> Caused by: java.nio.file.FileSystemException: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/build/core/test/J2/temp/lucene.index.TestDemoParallelLeafReader_58D85E1FF5DF2279-001/tempDir-002/segs/5pq1dc4s6dld9ab6oe3gbjf4r_47/_0_Asserting_0.dvd: Too many open files
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleLimitFS.onOpen(HandleLimitFS.java:48)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.callOpenHook(HandleTrackingFS.java:81)
   [junit4]   2> 	at org.apache.lucene.mockfile.HandleTrackingFS.newOutputStream(HandleTrackingFS.java:160)
   [junit4]   2> 	at org.apache.lucene.mockfile.FilterFileSystemProvider.newOutputStream(FilterFileSystemProvider.java:197)
   [junit4]   2> 	at java.nio.file.Files.newOutputStream(Files.java:216)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:410)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory$FSIndexOutput.<init>(FSDirectory.java:406)
   [junit4]   2> 	at org.apache.lucene.store.FSDirectory.createOutput(FSDirectory.java:254)
   [junit4]   2> 	at org.apache.lucene.store.MockDirectoryWrapper.createOutput(MockDirectoryWrapper.java:672)
   [junit4]   2> 	at org.apache.lucene.store.LockValidatingDirectoryWrapper.createOutput(LockValidatingDirectoryWrapper.java:44)
   [junit4]   2> 	at org.apache.lucene.store.TrackingDirectoryWrapper.createOutput(TrackingDirectoryWrapper.java:43)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesConsumer.<init>(Lucene80DocValuesConsumer.java:79)
   [junit4]   2> 	at org.apache.lucene.codecs.lucene80.Lucene80DocValuesFormat.fieldsConsumer(Lucene80DocValuesFormat.java:161)
   [junit4]   2> 	at org.apache.lucene.codecs.asserting.AssertingDocValuesFormat.fieldsConsumer(AssertingDocValuesFormat.java:55)
   [junit4]   2> 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:227)
   [junit4]   2> 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.getInstance(PerFieldDocValuesFormat.java:163)
   [junit4]   2> 	at org.apache.lucene.codecs.perfield.PerFieldDocValuesFormat$FieldsWriter.addNumericField(PerFieldDocValuesFormat.java:109)
   [junit4]   2> 	at org.apache.lucene.index.NumericDocValuesWriter.flush(NumericDocValuesWriter.java:108)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.writeDocValues(DefaultIndexingChain.java:345)
   [junit4]   2> 	at org.apache.lucene.index.DefaultIndexingChain.flush(DefaultIndexingChain.java:225)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriterPerThread.flush(DocumentsWriterPerThread.java:350)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.doFlush(DocumentsWriter.java:476)
   [junit4]   2> 	at org.apache.lucene.index.DocumentsWriter.flushAllThreads(DocumentsWriter.java:656)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3911)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3886)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2089)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.forceMerge(IndexWriter.java:2069)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$2.reindex(TestDemoParallelLeafReader.java:747)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getParallelLeafReader(TestDemoParallelLeafReader.java:405)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader.getCurrentReader(TestDemoParallelLeafReader.java:163)
   [junit4]   2> 	at org.apache.lucene.index.TestDemoParallelLeafReader$ReindexingReader$ReindexingMergePolicy$ReindexingOneMerge.wrapForMerge(TestDemoParallelLeafReader.java:533)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4713)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4363)
   [junit4]   2> 	at org.apache.lucene.index.IndexWriter$IndexWriterMergeSource.merge(IndexWriter.java:5922)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:626)
   [junit4]   2> 	at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:684)
   [junit4]   2> 
   [junit4]   2> mar. 08, 2021 7:50:19 PM 

[...truncated too long message...]

lrTestCaseJ4 ####initCore end
   [junit4]   2> 422398 INFO  (searcherExecutor-4901-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 422401 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.h.SnapShooter Creating backup snapshot empty_backup1 at file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestSnapshotCoreBackup_B03222CD88624505-001/tempDir-003/
   [junit4]   2> 422402 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.h.SnapShooter Done creating backup snapshot: empty_backup1 into file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestSnapshotCoreBackup_B03222CD88624505-001/tempDir-003/snapshot.empty_backup1
   [junit4]   2> 422403 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.c.s.SolrSnapshotMetaDataManager Creating the snapshot named empty_snapshotA for core collection1 associated with index commit with generation 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestSnapshotCoreBackup_B03222CD88624505-001/data-dir-41-001/index/
   [junit4]   2> 422404 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{add=[1]} 0 0
   [junit4]   2> 422404 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.h.SnapShooter Creating backup snapshot empty_backup2 at file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestSnapshotCoreBackup_B03222CD88624505-001/tempDir-003/
   [junit4]   2> 422404 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.h.SnapShooter Done creating backup snapshot: empty_backup2 into file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestSnapshotCoreBackup_B03222CD88624505-001/tempDir-003/snapshot.empty_backup2
   [junit4]   2> 422405 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.c.s.SolrSnapshotMetaDataManager Creating the snapshot named empty_snapshotB for core collection1 associated with index commit with generation 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestSnapshotCoreBackup_B03222CD88624505-001/data-dir-41-001/index/
   [junit4]   2> 422414 INFO  (searcherExecutor-4901-thread-1-processing-x:collection1) [    x:collection1 ] o.a.s.c.SolrCore [collection1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 422415 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=null path=null params={}{commit=} 0 9
   [junit4]   2> 422416 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.h.SnapShooter Creating backup snapshot empty_backup_from_empty_snapshotA at file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestSnapshotCoreBackup_B03222CD88624505-001/tempDir-003/
   [junit4]   2> 422417 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.h.SnapShooter Done creating backup snapshot: empty_backup_from_empty_snapshotA into file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestSnapshotCoreBackup_B03222CD88624505-001/tempDir-003/snapshot.empty_backup_from_empty_snapshotA
   [junit4]   2> 422418 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.h.SnapShooter Creating backup snapshot empty_backup_from_empty_snapshotB at file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestSnapshotCoreBackup_B03222CD88624505-001/tempDir-003/
   [junit4]   2> 422418 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [    x:collection1 ] o.a.s.h.SnapShooter Done creating backup snapshot: empty_backup_from_empty_snapshotB into file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestSnapshotCoreBackup_B03222CD88624505-001/tempDir-003/snapshot.empty_backup_from_empty_snapshotB
   [junit4]   2> 422418 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [     ] o.a.s.SolrTestCaseJ4 ###deleteCore
   [junit4]   2> 422418 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1065068593
   [junit4]   2> 422419 INFO  (coreCloseExecutor-4907-thread-1) [     ] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2a0ab89e
   [junit4]   2> 422419 INFO  (coreCloseExecutor-4907-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1 tag=SolrCore@2a0ab89e
   [junit4]   2> 422419 INFO  (coreCloseExecutor-4907-thread-1) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7caf3bb3: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1/com.codahale.metrics.MetricRegistry@52aab2f2
   [junit4]   2> 422452 INFO  (coreCloseExecutor-4907-thread-1) [     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 422478 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 422478 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5e2395b0: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@70835426
   [junit4]   2> 422484 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
   [junit4]   2> 422484 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@69bd631a: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@672a3782
   [junit4]   2> 422487 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 422487 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3041fb41: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@60ae4181
   [junit4]   2> 422622 INFO  (TEST-TestSnapshotCoreBackup.testBackupBeforeFirstCommit-seed#[B03222CD88624505]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testBackupBeforeFirstCommit
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J0/temp/solr.handler.TestSnapshotCoreBackup_B03222CD88624505-001
   [junit4]   2> NOTE: test params are: codec=SimpleText, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=zh-CN, timezone=Australia/LHI
   [junit4]   2> NOTE: Linux 4.15.0-112-generic amd64/Oracle Corporation 1.8.0_281 (64-bit)/cpus=4,threads=1,free=266966440,total=525336576
   [junit4]   2> NOTE: All tests run in this JVM: [SolrSlf4jReporterTest, CdcrUpdateLogTest, LeaderElectionContextKeyTest, SearchRateTriggerIntegrationTest, ShardRoutingCustomTest, PreAnalyzedUpdateProcessorTest, TestReqParamsAPI, TestPhraseSuggestions, IgnoreLargeDocumentProcessorFactoryTest, TestQuerySenderListener, AnalyticsMergeStrategyTest, DistribCursorPagingTest, AtomicUpdatesTest, FileUtilsTest, OverseerTest, TestAuthorizationFramework, UnloadDistributedZkTest, TestDelegationWithHadoopAuth, TestSolrCoreSnapshots, ReplaceNodeTest, UtilsToolTest, TestSchemaField, TestMacroExpander, DistributedQueryComponentCustomSortTest, TestCustomStream, TermVectorComponentTest, DistributedVersionInfoTest, SegmentsInfoRequestHandlerTest, OpenExchangeRatesOrgProviderTest, TestTextField, PrimitiveFieldTypeTest, TestFileDictionaryLookup, AssignTest, ReturnFieldsTest, TestRTimerTree, OverseerSolrResponseTest, CustomHighlightComponentTest, CoreAdminCreateDiscoverTest, TestNumericTokenStream, AutoAddReplicasPlanActionTest, RequestHandlersTest, TestManagedSchemaAPI, TestSnapshotCoreBackup]
   [junit4] Completed [114/934 (1!)] on J0 in 7.49s, 4 tests, 2 errors <<< FAILURES!

[...truncated 53611 lines...]

[JENKINS] Lucene » Lucene-Solr-NightlyTests-8.x - Build # 203 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-NightlyTests-8.x/203/

All tests passed

Build Log:
[...truncated 12196 lines...]
    [javac] Compiling 781 source files to /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-solrj/classes/java
    [javac] /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/solrj/src/java/org/apache/solr/client/solrj/impl/PreemptiveBasicAuthClientBuilderFactory.java:150: error: cannot find symbol
    [javac]         try (BufferedReader reader = Files.newBufferedReader(Path.of(configFile), StandardCharsets.UTF_8)) {
    [javac]                                                                  ^
    [javac]   symbol:   method of(String)
    [javac]   location: interface Path
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] 1 error

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:652: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:578: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:59: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build.xml:231: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/common-build.xml:550: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/common-build.xml:498: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/common-build.xml:393: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/common-build.xml:413: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/common-build.xml:581: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/common-build.xml:2083: Compile failed; see the compiler error output for details.

Total time: 214 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
[Checks API] No suitable checks publisher found.
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene » Lucene-Solr-NightlyTests-8.x - Build # 202 - Failure!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-NightlyTests-8.x/202/

All tests passed

Build Log:
[...truncated 12140 lines...]
    [javac] Compiling 781 source files to /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-solrj/classes/java
    [javac] /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/solrj/src/java/org/apache/solr/client/solrj/impl/PreemptiveBasicAuthClientBuilderFactory.java:150: error: cannot find symbol
    [javac]         try (BufferedReader reader = Files.newBufferedReader(Path.of(configFile), StandardCharsets.UTF_8)) {
    [javac]                                                                  ^
    [javac]   symbol:   method of(String)
    [javac]   location: interface Path
    [javac] Note: Some input files use or override a deprecated API.
    [javac] Note: Recompile with -Xlint:deprecation for details.
    [javac] Note: Some input files use unchecked or unsafe operations.
    [javac] Note: Recompile with -Xlint:unchecked for details.
    [javac] 1 error

BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:652: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:578: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:59: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build.xml:231: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/common-build.xml:550: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/common-build.xml:498: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/common-build.xml:393: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/common-build.xml:413: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/common-build.xml:581: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/lucene/common-build.xml:2083: Compile failed; see the compiler error output for details.

Total time: 180 minutes 11 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene » Lucene-Solr-NightlyTests-8.x - Build # 201 - Still unstable!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-NightlyTests-8.x/201/

2 tests failed.
FAILED:  org.apache.solr.cloud.LeaderTragicEventTest.testLeaderFailsOver

Error Message:
Error from server at https://127.0.0.1:41937/solr: Underlying core creation failed while creating collection: testLeaderFailsOver

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:41937/solr: Underlying core creation failed while creating collection: testLeaderFailsOver
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:681)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:266)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:369)
	at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:297)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1171)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:934)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:866)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:214)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:231)
	at org.apache.solr.cloud.LeaderTragicEventTest.testLeaderFailsOver(LeaderTragicEventTest.java:80)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.cloud.MoveReplicaTest.test

Error Message:
expected:<100> but was:<44>

Stack Trace:
java.lang.AssertionError: expected:<100> but was:<44>
	at __randomizedtesting.SeedInfo.seed([ED34A8CBFC0B7E74:6560971152F7138C]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.junit.Assert.failNotEquals(Assert.java:835)
	at org.junit.Assert.assertEquals(Assert.java:647)
	at org.junit.Assert.assertEquals(Assert.java:633)
	at org.apache.solr.cloud.MoveReplicaTest.test(MoveReplicaTest.java:235)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14380 lines...]
   [junit4] Suite: org.apache.solr.cloud.MoveReplicaTest
   [junit4]   2> 1905975 INFO  (SUITE-MoveReplicaTest-seed#[ED34A8CBFC0B7E74]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 1905975 INFO  (SUITE-MoveReplicaTest-seed#[ED34A8CBFC0B7E74]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 1905975 INFO  (SUITE-MoveReplicaTest-seed#[ED34A8CBFC0B7E74]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/data-dir-134-001
   [junit4]   2> 1905975 WARN  (SUITE-MoveReplicaTest-seed#[ED34A8CBFC0B7E74]-worker) [     ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=23 numCloses=23
   [junit4]   2> 1905975 INFO  (SUITE-MoveReplicaTest-seed#[ED34A8CBFC0B7E74]-worker) [     ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 1905982 INFO  (SUITE-MoveReplicaTest-seed#[ED34A8CBFC0B7E74]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 1905982 INFO  (SUITE-MoveReplicaTest-seed#[ED34A8CBFC0B7E74]-worker) [     ] o.a.s.c.SolrCloudTestCase Using per-replica state
   [junit4]   2> 1905990 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testFailedMove
   [junit4]   2> 1905993 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001
   [junit4]   2> 1905993 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1905994 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 1905994 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1905995 WARN  (ZkTestServer Run Thread) [     ] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0.
   [junit4]   2> 1906094 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.ZkTestServer start zk server on port: 42475
   [junit4]   2> 1906094 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:42475
   [junit4]   2> 1906094 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:42475
   [junit4]   2> 1906094 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 42475
   [junit4]   2> 1906097 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1906104 INFO  (zkConnectionManagerCallback-16874-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1906104 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1906110 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1906112 INFO  (zkConnectionManagerCallback-16876-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1906112 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1906113 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1906116 INFO  (zkConnectionManagerCallback-16878-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1906116 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1906230 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1906230 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1906230 INFO  (jetty-launcher-16879-thread-1) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 1906233 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1906233 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1906233 INFO  (jetty-launcher-16879-thread-2) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 1906233 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1906233 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1906233 INFO  (jetty-launcher-16879-thread-4) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 1906234 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 1906234 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 1906234 INFO  (jetty-launcher-16879-thread-3) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 1906244 INFO  (jetty-launcher-16879-thread-2) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1906244 INFO  (jetty-launcher-16879-thread-2) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1906244 INFO  (jetty-launcher-16879-thread-2) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1906252 INFO  (jetty-launcher-16879-thread-4) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1906252 INFO  (jetty-launcher-16879-thread-4) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1906252 INFO  (jetty-launcher-16879-thread-4) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1906252 INFO  (jetty-launcher-16879-thread-4) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@41fa9d89{/solr,null,AVAILABLE}
   [junit4]   2> 1906252 INFO  (jetty-launcher-16879-thread-4) [     ] o.e.j.s.AbstractConnector Started ServerConnector@3ff5cab6{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:46035}
   [junit4]   2> 1906252 INFO  (jetty-launcher-16879-thread-4) [     ] o.e.j.s.Server Started @1906357ms
   [junit4]   2> 1906252 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, zkHost=127.0.0.1:42475/solr, hostPort=46035}
   [junit4]   2> 1906252 ERROR (jetty-launcher-16879-thread-4) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1906252 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1906253 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 1906253 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1906253 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 1906253 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-05T15:37:42.432Z
   [junit4]   2> 1906253 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1906319 INFO  (jetty-launcher-16879-thread-1) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1906319 INFO  (jetty-launcher-16879-thread-1) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1906320 INFO  (jetty-launcher-16879-thread-1) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1906320 INFO  (jetty-launcher-16879-thread-1) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@79da793e{/solr,null,AVAILABLE}
   [junit4]   2> 1906320 INFO  (jetty-launcher-16879-thread-1) [     ] o.e.j.s.AbstractConnector Started ServerConnector@422180{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:34111}
   [junit4]   2> 1906320 INFO  (jetty-launcher-16879-thread-1) [     ] o.e.j.s.Server Started @1906425ms
   [junit4]   2> 1906320 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, zkHost=127.0.0.1:42475/solr, hostPort=34111}
   [junit4]   2> 1906320 INFO  (jetty-launcher-16879-thread-2) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@bee7259{/solr,null,AVAILABLE}
   [junit4]   2> 1906321 ERROR (jetty-launcher-16879-thread-1) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1906321 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1906321 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 1906321 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1906321 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 1906321 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-05T15:37:42.500Z
   [junit4]   2> 1906323 INFO  (jetty-launcher-16879-thread-3) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1906323 INFO  (jetty-launcher-16879-thread-3) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1906323 INFO  (jetty-launcher-16879-thread-3) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1906323 INFO  (jetty-launcher-16879-thread-3) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@9b10044{/solr,null,AVAILABLE}
   [junit4]   2> 1906324 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1906324 INFO  (zkConnectionManagerCallback-16881-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1906324 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1906325 INFO  (jetty-launcher-16879-thread-2) [     ] o.e.j.s.AbstractConnector Started ServerConnector@5aff484b{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:36945}
   [junit4]   2> 1906325 INFO  (jetty-launcher-16879-thread-2) [     ] o.e.j.s.Server Started @1906429ms
   [junit4]   2> 1906325 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, zkHost=127.0.0.1:42475/solr, hostPort=36945}
   [junit4]   2> 1906325 ERROR (jetty-launcher-16879-thread-2) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1906325 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1906325 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 1906325 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1906325 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 1906325 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-05T15:37:42.504Z
   [junit4]   2> 1906326 INFO  (jetty-launcher-16879-thread-3) [     ] o.e.j.s.AbstractConnector Started ServerConnector@56a03e2d{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:40907}
   [junit4]   2> 1906326 INFO  (jetty-launcher-16879-thread-3) [     ] o.e.j.s.Server Started @1906430ms
   [junit4]   2> 1906326 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, zkHost=127.0.0.1:42475/solr, hostPort=40907}
   [junit4]   2> 1906326 ERROR (jetty-launcher-16879-thread-3) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1906326 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1906326 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 1906326 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1906326 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 1906326 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-05T15:37:42.505Z
   [junit4]   2> 1906326 INFO  (zkConnectionManagerCallback-16883-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1906326 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1906327 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1906327 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1906327 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1906327 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1906330 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1906330 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1906332 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1906333 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1906333 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1906334 INFO  (zkConnectionManagerCallback-16887-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1906334 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1906334 INFO  (zkConnectionManagerCallback-16885-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1906334 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1906335 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1906335 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1906335 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1906338 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1906338 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1906338 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1906338 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1906339 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1906340 WARN  (jetty-launcher-16879-thread-3-SendThread(127.0.0.1:42475)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103b9f6b7640005.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103b9f6b7640005, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103b9f6b7640005, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 1906341 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1906443 WARN  (jetty-launcher-16879-thread-2-SendThread(127.0.0.1:42475)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103b9f6b7640006.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103b9f6b7640006, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103b9f6b7640006, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 1906722 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1906723 WARN  (jetty-launcher-16879-thread-4) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@64e02870[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1906723 WARN  (jetty-launcher-16879-thread-4) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@64e02870[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1906725 WARN  (jetty-launcher-16879-thread-4) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5f62a450[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1906725 WARN  (jetty-launcher-16879-thread-4) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5f62a450[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1906727 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42475/solr
   [junit4]   2> 1906734 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1906852 INFO  (zkConnectionManagerCallback-16907-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1906852 INFO  (jetty-launcher-16879-thread-4) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1906893 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1906894 WARN  (jetty-launcher-16879-thread-3) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@189ffc69[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1906894 WARN  (jetty-launcher-16879-thread-3) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@189ffc69[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1906896 WARN  (jetty-launcher-16879-thread-3) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@775cb18f[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1906896 WARN  (jetty-launcher-16879-thread-3) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@775cb18f[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1906897 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42475/solr
   [junit4]   2> 1906898 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1906899 INFO  (zkConnectionManagerCallback-16915-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1906899 INFO  (jetty-launcher-16879-thread-3) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1906954 INFO  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1906955 INFO  (zkConnectionManagerCallback-16917-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1906955 INFO  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1907013 INFO  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46035_solr
   [junit4]   2> 1907013 INFO  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.c.Overseer Overseer (id=73106488257937417-127.0.0.1:46035_solr-n_0000000000) starting
   [junit4]   2> 1907017 INFO  (OverseerStateUpdate-73106488257937417-127.0.0.1:46035_solr-n_0000000000) [n:127.0.0.1:46035_solr     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:46035_solr
   [junit4]   2> 1907023 INFO  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46035_solr
   [junit4]   2> 1907025 INFO  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1907026 WARN  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1907035 INFO  (zkCallback-16916-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1907043 INFO  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1907056 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1907057 INFO  (zkConnectionManagerCallback-16930-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1907057 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1907061 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1907064 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:40907_solr as DOWN
   [junit4]   2> 1907065 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1907065 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40907_solr
   [junit4]   2> 1907066 INFO  (zkCallback-16916-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1907067 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1907068 WARN  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1907198 INFO  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907203 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1907207 INFO  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907207 INFO  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907208 INFO  (jetty-launcher-16879-thread-4) [n:127.0.0.1:46035_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node3
   [junit4]   2> 1907220 INFO  (zkCallback-16929-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1907223 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907233 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907233 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907235 INFO  (jetty-launcher-16879-thread-3) [n:127.0.0.1:40907_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node4
   [junit4]   2> 1907312 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1907313 WARN  (jetty-launcher-16879-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@60c676d2[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1907313 WARN  (jetty-launcher-16879-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@60c676d2[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1907315 WARN  (jetty-launcher-16879-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5f621d52[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1907315 WARN  (jetty-launcher-16879-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5f621d52[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1907317 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42475/solr
   [junit4]   2> 1907317 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1907318 INFO  (zkConnectionManagerCallback-16947-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1907318 INFO  (jetty-launcher-16879-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1907383 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 1907383 WARN  (jetty-launcher-16879-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@644c77af[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1907384 WARN  (jetty-launcher-16879-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@644c77af[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1907386 WARN  (jetty-launcher-16879-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3369e63[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1907386 WARN  (jetty-launcher-16879-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3369e63[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 1907388 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42475/solr
   [junit4]   2> 1907390 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1907390 INFO  (zkConnectionManagerCallback-16955-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1907390 INFO  (jetty-launcher-16879-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1907391 WARN  (jetty-launcher-16879-thread-2-SendThread(127.0.0.1:42475)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103b9f6b764000c.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103b9f6b764000c, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103b9f6b764000c, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 1907424 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1907425 INFO  (zkConnectionManagerCallback-16957-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1907425 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1907443 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1907449 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:34111_solr as DOWN
   [junit4]   2> 1907451 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1907451 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34111_solr
   [junit4]   2> 1907452 INFO  (zkCallback-16929-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1907452 INFO  (zkCallback-16916-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1907453 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1907454 WARN  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1907455 INFO  (zkCallback-16956-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 1907466 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1907485 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907504 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907504 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907505 INFO  (jetty-launcher-16879-thread-1) [n:127.0.0.1:34111_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node1
   [junit4]   2> 1907521 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1907525 INFO  (zkConnectionManagerCallback-16967-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1907525 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1907535 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
   [junit4]   2> 1907547 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:36945_solr as DOWN
   [junit4]   2> 1907548 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1907548 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36945_solr
   [junit4]   2> 1907550 INFO  (zkCallback-16916-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1907550 INFO  (zkCallback-16929-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1907551 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 1907551 WARN  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 1907555 INFO  (zkCallback-16956-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1907555 INFO  (zkCallback-16966-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
   [junit4]   2> 1907564 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1907585 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907596 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907596 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1907597 INFO  (jetty-launcher-16879-thread-2) [n:127.0.0.1:36945_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node2
   [junit4]   2> 1907626 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=4
   [junit4]   2> 1907628 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 1907630 INFO  (zkConnectionManagerCallback-16980-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1907630 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 1907636 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
   [junit4]   2> 1907639 INFO  (TEST-MoveReplicaTest.testFailedMove-seed#[ED34A8CBFC0B7E74]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42475/solr ready
   [junit4]   2> 1907660 INFO  (qtp1483197265-39046) [n:127.0.0.1:40907_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=4
   [junit4]   2> 1907664 INFO  (OverseerCollectionConfigSetProcessor-73106488257937417-127.0.0.1:46035_solr-n_0000000000) [n:127.0.0.1:46035_solr     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1907664 INFO  (OverseerThreadFactory-16924-thread-2-processing-n:127.0.0.1:46035_solr) [n:127.0.0.1:46035_solr     ] o.a.s.c.a.c.CreateCollectionCmd Create collection MoveReplicaTest_failed_coll_true
   [junit4]   2> 1907773 INFO  (OverseerStateUpdate-73106488257937417-127.0.0.1:46035_solr-n_0000000000) [n:127.0.0.1:46035_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "core":"MoveReplicaTest_failed_coll_true_shard1_replica_n1",
   [junit4]   2>   "node_name":"127.0.0.1:40907_solr",
   [junit4]   2>   "base_url":"http://127.0.0.1:40907/solr",
   [junit4]   2>   "collection":"MoveReplicaTest_failed_coll_true",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1907775 INFO  (OverseerStateUpdate-73106488257937417-127.0.0.1:46035_solr-n_0000000000) [n:127.0.0.1:46035_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "core":"MoveReplicaTest_failed_coll_true_shard1_replica_p2",
   [junit4]   2>   "node_name":"127.0.0.1:46035_solr",
   [junit4]   2>   "base_url":"http://127.0.0.1:46035/solr",
   [junit4]   2>   "collection":"MoveReplicaTest_failed_coll_true",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "type":"PULL",
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1907777 INFO  (OverseerStateUpdate-73106488257937417-127.0.0.1:46035_solr-n_0000000000) [n:127.0.0.1:46035_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "core":"MoveReplicaTest_failed_coll_true_shard2_replica_n4",
   [junit4]   2>   "node_name":"127.0.0.1:34111_solr",
   [junit4]   2>   "base_url":"http://127.0.0.1:34111/solr",
   [junit4]   2>   "collection":"MoveReplicaTest_failed_coll_true",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1907779 INFO  (OverseerStateUpdate-73106488257937417-127.0.0.1:46035_solr-n_0000000000) [n:127.0.0.1:46035_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "core":"MoveReplicaTest_failed_coll_true_shard2_replica_p6",
   [junit4]   2>   "node_name":"127.0.0.1:36945_solr",
   [junit4]   2>   "base_url":"http://127.0.0.1:36945/solr",
   [junit4]   2>   "collection":"MoveReplicaTest_failed_coll_true",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "type":"PULL",
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1907983 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr    x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=MoveReplicaTest_failed_coll_true_shard1_replica_p2&action=CREATE&numShards=2&collection=MoveReplicaTest_failed_coll_true&shard=shard1&wt=javabin&version=2&replicaType=PULL
   [junit4]   2> 1907983 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr    x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=MoveReplicaTest_failed_coll_true_shard1_replica_n1&action=CREATE&numShards=2&collection=MoveReplicaTest_failed_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1907983 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr    x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1907984 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr    x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=MoveReplicaTest_failed_coll_true_shard2_replica_n4&action=CREATE&numShards=2&collection=MoveReplicaTest_failed_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 1907984 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr    x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=MoveReplicaTest_failed_coll_true_shard2_replica_p6&action=CREATE&numShards=2&collection=MoveReplicaTest_failed_coll_true&shard=shard2&wt=javabin&version=2&replicaType=PULL
   [junit4]   2> 1907997 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 1908001 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 1908001 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 1908011 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 1908015 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=cloud-dynamic
   [junit4]   2> 1908017 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.s.IndexSchema Schema name=cloud-dynamic
   [junit4]   2> 1908019 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 1908019 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaTest_failed_coll_true_shard1_replica_n1' using configuration from configset conf1, trusted=true
   [junit4]   2> 1908019 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.s.IndexSchema Schema name=cloud-dynamic
   [junit4]   2> 1908019 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.MoveReplicaTest_failed_coll_true.shard1.replica_n1' (registry 'solr.core.MoveReplicaTest_failed_coll_true.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1908019 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.SolrCore [[MoveReplicaTest_failed_coll_true_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node4/MoveReplicaTest_failed_coll_true_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node4/MoveReplicaTest_failed_coll_true_shard1_replica_n1/data/]
   [junit4]   2> 1908020 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.s.IndexSchema Schema name=cloud-dynamic
   [junit4]   2> 1908020 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 1908020 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaTest_failed_coll_true_shard1_replica_p2' using configuration from configset conf1, trusted=true
   [junit4]   2> 1908020 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.MoveReplicaTest_failed_coll_true.shard1.replica_p2' (registry 'solr.core.MoveReplicaTest_failed_coll_true.shard1.replica_p2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1908020 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.c.SolrCore [[MoveReplicaTest_failed_coll_true_shard1_replica_p2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node3/MoveReplicaTest_failed_coll_true_shard1_replica_p2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node3/MoveReplicaTest_failed_coll_true_shard1_replica_p2/data/]
   [junit4]   2> 1908021 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 1908021 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaTest_failed_coll_true_shard2_replica_n4' using configuration from configset conf1, trusted=true
   [junit4]   2> 1908022 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.MoveReplicaTest_failed_coll_true.shard2.replica_n4' (registry 'solr.core.MoveReplicaTest_failed_coll_true.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1908022 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
   [junit4]   2> 1908022 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaTest_failed_coll_true_shard2_replica_p6' using configuration from configset conf1, trusted=true
   [junit4]   2> 1908022 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.SolrCore [[MoveReplicaTest_failed_coll_true_shard2_replica_n4] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node1/MoveReplicaTest_failed_coll_true_shard2_replica_n4], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node1/MoveReplicaTest_failed_coll_true_shard2_replica_n4/data/]
   [junit4]   2> 1908022 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.MoveReplicaTest_failed_coll_true.shard2.replica_p6' (registry 'solr.core.MoveReplicaTest_failed_coll_true.shard2.replica_p6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@20273db3
   [junit4]   2> 1908022 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.c.SolrCore [[MoveReplicaTest_failed_coll_true_shard2_replica_p6] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node2/MoveReplicaTest_failed_coll_true_shard2_replica_p6], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaTest_ED34A8CBFC0B7E74-001/tempDir-001/node2/MoveReplicaTest_failed_coll_true_shard2_replica_p6/data/]
   [junit4]   2> 1908122 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1908122 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1908123 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1908123 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1908125 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1908126 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1908126 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1908126 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693406896382279680
   [junit4]   2> 1908127 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1908127 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1908128 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1908128 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1908129 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1908129 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1908130 INFO  (searcherExecutor-16982-thread-1-processing-n:127.0.0.1:40907_solr x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.SolrCore [MoveReplicaTest_failed_coll_true_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1908130 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1908131 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1908131 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1908132 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1908132 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1908132 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1908132 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693406896388571136
   [junit4]   2> 1908135 INFO  (searcherExecutor-16984-thread-1-processing-n:127.0.0.1:46035_solr x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.c.SolrCore [MoveReplicaTest_failed_coll_true_shard1_replica_p2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1908136 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/MoveReplicaTest_failed_coll_true/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 1908136 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1908136 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/MoveReplicaTest_failed_coll_true/leaders/shard1
   [junit4]   2> 1908136 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1908138 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.c.ZkController MoveReplicaTest_failed_coll_true_shard1_replica_p2 starting background replication from leader
   [junit4]   2> 1908138 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:01
   [junit4]   2> 1908138 INFO  (searcherExecutor-16986-thread-1-processing-n:127.0.0.1:34111_solr x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.SolrCore [MoveReplicaTest_failed_coll_true_shard2_replica_n4]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1908140 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 1908140 INFO  (qtp825369312-39028) [n:127.0.0.1:46035_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaTest_failed_coll_true_shard1_replica_p2 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1908141 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 1908146 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 1908146 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1908150 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/MoveReplicaTest_failed_coll_true/terms/shard2 to Terms{values={core_node7=0}, version=0}
   [junit4]   2> 1908150 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/MoveReplicaTest_failed_coll_true/leaders/shard2
   [junit4]   2> 1908150 INFO  (searcherExecutor-16988-thread-1-processing-n:127.0.0.1:36945_solr x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.c.SolrCore [MoveReplicaTest_failed_coll_true_shard2_replica_p6]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1908154 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1908154 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1908154 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40907/solr/MoveReplicaTest_failed_coll_true_shard1_replica_n1/
   [junit4]   2> 1908159 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1908161 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.SyncStrategy http://127.0.0.1:40907/solr/MoveReplicaTest_failed_coll_true_shard1_replica_n1/ has no replicas
   [junit4]   2> 1908161 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/MoveReplicaTest_failed_coll_true/leaders/shard1/leader after winning as /collections/MoveReplicaTest_failed_coll_true/leader_elect/shard1/election/73106488257937418-core_node3-n_0000000000
   [junit4]   2> 1908162 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.c.ZkController MoveReplicaTest_failed_coll_true_shard2_replica_p6 starting background replication from leader
   [junit4]   2> 1908162 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:01
   [junit4]   2> 1908164 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 1908164 INFO  (qtp1413000380-39019) [n:127.0.0.1:36945_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaTest_failed_coll_true_shard2_replica_p6 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 1908166 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40907/solr/MoveReplicaTest_failed_coll_true_shard1_replica_n1/ shard1
   [junit4]   2> 1908167 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1908167 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1908167 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34111/solr/MoveReplicaTest_failed_coll_true_shard2_replica_n4/
   [junit4]   2> 1908168 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 1908168 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.SyncStrategy http://127.0.0.1:34111/solr/MoveReplicaTest_failed_coll_true_shard2_replica_n4/ has no replicas
   [junit4]   2> 1908168 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/MoveReplicaTest_failed_coll_true/leaders/shard2/leader after winning as /collections/MoveReplicaTest_failed_coll_true/leader_elect/shard2/election/73106488257937421-core_node7-n_0000000000
   [junit4]   2> 1908174 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34111/solr/MoveReplicaTest_failed_coll_true_shard2_replica_n4/ shard2
   [junit4]   2> 1908277 INFO  (zkCallback-16956-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaTest_failed_coll_true/state.json] for collection [MoveReplicaTest_failed_coll_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1908277 INFO  (zkCallback-16966-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaTest_failed_coll_true/state.json] for collection [MoveReplicaTest_failed_coll_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1908277 INFO  (zkCallback-16916-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaTest_failed_coll_true/state.json] for collection [MoveReplicaTest_failed_coll_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1908277 INFO  (zkCallback-16929-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaTest_failed_coll_true/state.json] for collection [MoveReplicaTest_failed_coll_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1908278 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1908278 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr c:MoveReplicaTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaTest_failed_coll_true_shard2_replica_n4 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1908278 INFO  (zkCallback-16916-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaTest_failed_coll_true/state.json] for collection [MoveReplicaTest_failed_coll_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1908279 INFO  (indexFetcher-17004-thread-1) [     ] o.a.s.h.IndexFetcher Replica core_node3 is leader but it's state is down, skipping replication
   [junit4]   2> 1908279 INFO  (indexFetcher-17006-thread-1) [     ] o.a.s.h.IndexFetcher Replica core_node7 is leader but it's state is down, skipping replication
   [junit4]   2> 1908290 INFO  (qtp1483197265-39049) [n:127.0.0.1:40907_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=MoveReplicaTest_failed_coll_true_shard1_replica_n1&action=CREATE&numShards=2&collection=MoveReplicaTest_failed_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=307
   [junit4]   2> 1908293 INFO  (qtp243650080-39038) [n:127.0.0.1:34111_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=MoveReplicaTest_failed_coll_true_shard2_replica_n4&action=CREATE&numShards=2&collection=MoveReplicaTest_failed_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=309
   [junit4]   2> 1908388 INFO  (zkCallback-16956-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaTest_failed_coll_true/state.json] for collection [MoveReplicaTest_failed_coll_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1908388 INFO  (zkCallback-16929-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaTest_failed_coll_true/state.json] for collection [MoveReplicaTest_failed_coll_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1908388 INFO  (zkCallback-16929-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaTest_failed_coll_true/state.json] for collection [MoveReplicaTest_failed_coll_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1908388 INFO  (zkCallback-16956-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaTest_failed_coll_true/state.json] for collection [MoveReplicaTest_failed_coll_true] has occurred - updating... (live nodes size: [4])
   [junit4]   2> 1909140 INFO  (indexFetcher-17004-thread-1) [     ] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication
   [junit4]   2> 1909146 INFO  (indexFetcher-17004-thread-1) [     ] o.a.s.h.IndexFetcher Updated leaderUrl to http://127.0.0.1:40907/solr/MoveReplicaTest_failed_coll_true_shard1_replica_n1/
   [junit4]   2> 1909147 INFO  (qtp1483197265-39046) [n:127.0.0.1:40907_solr c:MoveReplicaTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaTest_failed_coll_true_shard1_replica_n1 ] o.a.s.c.S.Request [MoveReplicaTest_failed_coll_true_shard1_replica_n1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 1909148 INFO  (indexFetcher-17004-thread-1) [     ] o.a.s.h.IndexFetcher Leader's generation: 1
   [junit4]   2> 1909148 INFO  (indexFetcher-17004-thread-1) [     ] o.a.s.h.IndexFetcher Leader's version: 0
   [junit4]   2> 1909148 INFO  (indexFetcher-17004-thread-1) [     ] o.a.s.h.IndexFetcher Follower's generation: 1
   [junit4]   2> 1909148 INFO  (indexFetcher-17004-thread-1) [     ] o.a.s.h.IndexFetcher Follower's version: 0
   [junit4]   2> 1909148 INFO  (indexFetcher-17004-thread-1) [     ] o.a.s.h.IndexFetcher New index in Leader. Deleting mine...
   [junit4]   2> 1909156 INFO  (searcherExecutor-16984-thread-1) [     ] o.a.s.c.SolrCore [MoveReplicaTest_failed_coll_true_shard1_replica_p2]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 1909164 INFO  (indexFetcher-17006-thread-1) [     ] o.a.s.h.Ind

[...truncated too long message...]

Publish node=127.0.0.1:41937_solr as DOWN
   [junit4]   2> 3961239 INFO  (jetty-closer-69069-thread-2) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=630866602
   [junit4]   2> 3961239 INFO  (jetty-closer-69069-thread-2) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:43521_solr
   [junit4]   2> 3961240 INFO  (jetty-closer-69069-thread-2) [     ] o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 3961240 INFO  (jetty-closer-69069-thread-2) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:43521_solr as DOWN
   [junit4]   2> 3961271 INFO  (jetty-closer-69069-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 3961271 INFO  (jetty-closer-69069-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
   [junit4]   2> 3961271 INFO  (jetty-closer-69069-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 3961271 INFO  (jetty-closer-69069-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
   [junit4]   2> 3961304 INFO  (jetty-closer-69069-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 3961304 INFO  (jetty-closer-69069-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
   [junit4]   2> 3961304 INFO  (jetty-closer-69069-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 3961305 INFO  (jetty-closer-69069-thread-1) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
   [junit4]   2> 3961314 INFO  (closeThreadPool-69079-thread-2) [     ] o.a.s.c.Overseer Overseer (id=73106621913104390-127.0.0.1:41937_solr-n_0000000000) closing
   [junit4]   2> 3961328 INFO  (OverseerAutoScalingTriggerThread-73106621913104390-127.0.0.1:41937_solr-n_0000000000) [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 3961329 INFO  (OverseerStateUpdate-73106621913104390-127.0.0.1:41937_solr-n_0000000000) [n:127.0.0.1:41937_solr     ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41937_solr
   [junit4]   2> 3961336 WARN  (jetty-launcher-68966-thread-2-SendThread(127.0.0.1:41579)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103ba15d5dc0006.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103ba15d5dc0006, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103ba15d5dc0006, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 3961336 INFO  (closeThreadPool-69079-thread-1) [     ] o.a.s.c.Overseer Overseer (id=73106621913104390-127.0.0.1:41937_solr-n_0000000000) closing
   [junit4]   2> 3961436 INFO  (jetty-closer-69069-thread-1) [     ] o.a.s.c.Overseer Overseer (id=73106621913104390-127.0.0.1:41937_solr-n_0000000000) closing
   [junit4]   2> 3961438 INFO  (jetty-closer-69069-thread-1) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@57dd1379{SSL, (ssl, http/1.1)}{127.0.0.1:0}
   [junit4]   2> 3961439 INFO  (jetty-closer-69069-thread-1) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@345ebd3e{/solr,null,STOPPED}
   [junit4]   2> 3961439 INFO  (jetty-closer-69069-thread-1) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 3961725 INFO  (jetty-closer-69069-thread-2) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@385f7b26{SSL, (ssl, http/1.1)}{127.0.0.1:43521}
   [junit4]   2> 3961725 INFO  (jetty-closer-69069-thread-2) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@61eed9af{/solr,null,STOPPED}
   [junit4]   2> 3961725 INFO  (jetty-closer-69069-thread-2) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 3961726 INFO  (SUITE-LeaderTragicEventTest-seed#[ED34A8CBFC0B7E74]-worker) [     ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 3961727 WARN  (SUITE-LeaderTragicEventTest-seed#[ED34A8CBFC0B7E74]-worker-SendThread(127.0.0.1:41579)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103ba15d5dc0000.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103ba15d5dc0000, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103ba15d5dc0000, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 3962162 WARN  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	9	/solr/collections/testOtherReplicasAreNotActive/terms/shard1
   [junit4]   2> 	7	/solr/collections/testOtherReplicasAreNotActive/state.json
   [junit4]   2> 	4	/solr/aliases.json
   [junit4]   2> 	4	/solr/collections/testLeaderFailsOver/state.json
   [junit4]   2> 	4	/solr/configs/config
   [junit4]   2> 	3	/solr/collections/testLeaderFailsOver/terms/shard1
   [junit4]   2> 	3	/solr/packages.json
   [junit4]   2> 	3	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	25	/solr/collections/testOtherReplicasAreNotActive/state.json
   [junit4]   2> 	5	/solr/collections/testLeaderFailsOver/state.json
   [junit4]   2> 	4	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/clusterprops.json
   [junit4]   2> 	2	/solr/overseer_elect/election/73106621913104390-127.0.0.1:41937_solr-n_0000000000
   [junit4]   2> 	2	/solr/collections/testOtherReplicasAreNotActive/leader_elect/shard1/election/73106621913104390-core_node4-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	42	/solr/overseer/queue
   [junit4]   2> 	16	/solr/collections
   [junit4]   2> 	14	/solr/collections/testOtherReplicasAreNotActive/state.json
   [junit4]   2> 	12	/solr/live_nodes
   [junit4]   2> 	12	/solr/overseer/collection-queue-work
   [junit4]   2> 	4	/solr/collections/testLeaderFailsOver/state.json
   [junit4]   2> 
   [junit4]   2> 3962164 INFO  (SUITE-LeaderTragicEventTest-seed#[ED34A8CBFC0B7E74]-worker) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:41579
   [junit4]   2> 3962164 INFO  (SUITE-LeaderTragicEventTest-seed#[ED34A8CBFC0B7E74]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:41579
   [junit4]   2> 3962164 INFO  (SUITE-LeaderTragicEventTest-seed#[ED34A8CBFC0B7E74]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 41579
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.LeaderTragicEventTest_ED34A8CBFC0B7E74-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene87): {}, docValues:{}, maxPointsInLeafNode=1369, maxMBSortInHeap=6.942865363977533, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=es-PY, timezone=Europe/Oslo
   [junit4]   2> NOTE: Linux 5.4.0-45-generic amd64/Oracle Corporation 1.8.0_281 (64-bit)/cpus=4,threads=1,free=183214864,total=523763712
   [junit4]   2> NOTE: All tests run in this JVM: [ZkStateWriterTest, TestUseDocValuesAsStored, CdcrBootstrapTest, PeerSyncTest, ByteBuffersDirectoryFactoryTest, DateMathParserTest, ReindexCollectionTest, TestExternalRoleRuleBasedAuthorizationPlugin, AssignBackwardCompatibilityTest, HdfsLockFactoryTest, TestConfigOverlay, ClusterStateTest, TestRecoveryHdfs, BJQFilterAccessibleTest, ExplicitHLLTest, TestSortByMinMaxFunction, TriggerEventQueueTest, NestedShardedAtomicUpdateTest, MetricsHistoryHandlerTest, RankFieldTest, HttpTriggerListenerTest, CdcrVersionReplicationTest, TestBM25SimilarityFactory, TestQuerySenderNoQuery, TestSimUtils, PurgeGraphTest, RequestLoggingTest, TestStressLiveNodes, TestCrossCoreJoin, TestSolrConfigHandlerCloud, ReplicationFactorTest, MultiDestinationAuditLoggerTest, TestJsonFacetsStatsParsing, TestClusterProperties, TestFieldCacheReopen, RoutingToNodesWithPropertiesTest, CustomCollectionTest, BasicFunctionalityTest, DocValuesNotIndexedTest, PolyFieldTest, PhrasesIdentificationComponentTest, TestLegacyField, TestValueSourceCache, TestLFUCache, TestFieldCollectionResource, PackageManagerCLITest, TestSimExtremeIndexing, TestReplicaProperties, TestSchemaNameResource, SignificantTermsQParserPluginTest, VersionInfoTest, ResourceLoaderTest, TestCollectionAPI, AnalyticsQueryTest, TestManagedSchemaThreadSafety, LeaderTragicEventTest]
   [junit4] Completed [615/934 (2!)] on J1 in 17.07s, 2 tests, 1 error <<< FAILURES!

[...truncated 51899 lines...]

[JENKINS] Lucene » Lucene-Solr-NightlyTests-8.x - Build # 200 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Error processing tokens: Error while parsing action 'Text/ZeroOrMore/FirstOf/Token/DelimitedToken/DelimitedToken_Action3' at input position (line 76, pos 4):
)"}
   ^

java.lang.OutOfMemoryError: Java heap space

[JENKINS] Lucene » Lucene-Solr-NightlyTests-8.x - Build # 199 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-NightlyTests-8.x/199/

1 tests failed.
FAILED:  org.apache.solr.cloud.cdcr.CdcrReplicationHandlerTest.testPartialReplicationAfterPeerSync

Error Message:
Error from server at https://127.0.0.1:37971/nt/b: ADDREPLICA failed to create replica

Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:37971/nt/b: ADDREPLICA failed to create replica
	at __randomizedtesting.SeedInfo.seed([C84D940E8CA4D6FC:DAEAB4E720D43DBA]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:681)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:266)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:369)
	at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:297)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1171)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:934)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:866)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:214)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:231)
	at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.startServers(BaseCdcrDistributedZkTest.java:601)
	at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.createSourceCollection(BaseCdcrDistributedZkTest.java:358)
	at org.apache.solr.cloud.cdcr.BaseCdcrDistributedZkTest.baseBefore(BaseCdcrDistributedZkTest.java:179)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:972)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1090)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1061)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13369 lines...]
   [junit4] Suite: org.apache.solr.cloud.cdcr.CdcrReplicationHandlerTest
   [junit4]   2> 329172 INFO  (SUITE-CdcrReplicationHandlerTest-seed#[C84D940E8CA4D6FC]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 329173 INFO  (SUITE-CdcrReplicationHandlerTest-seed#[C84D940E8CA4D6FC]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001/data-dir-27-001
   [junit4]   2> 329173 WARN  (SUITE-CdcrReplicationHandlerTest-seed#[C84D940E8CA4D6FC]-worker) [     ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=10 numCloses=10
   [junit4]   2> 329173 INFO  (SUITE-CdcrReplicationHandlerTest-seed#[C84D940E8CA4D6FC]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 329175 INFO  (SUITE-CdcrReplicationHandlerTest-seed#[C84D940E8CA4D6FC]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 329175 INFO  (SUITE-CdcrReplicationHandlerTest-seed#[C84D940E8CA4D6FC]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 329175 INFO  (SUITE-CdcrReplicationHandlerTest-seed#[C84D940E8CA4D6FC]-worker) [     ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /nt/b
   [junit4]   2> 329179 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 329179 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 329179 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 329180 WARN  (ZkTestServer Run Thread) [     ] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0.
   [junit4]   2> 329279 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer start zk server on port: 44881
   [junit4]   2> 329279 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:44881
   [junit4]   2> 329279 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:44881
   [junit4]   2> 329279 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 44881
   [junit4]   2> 329282 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 329285 INFO  (zkConnectionManagerCallback-1722-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 329285 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 329298 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 329303 INFO  (zkConnectionManagerCallback-1724-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 329303 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 329305 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-cdcr.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 329306 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 329308 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 329309 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 329309 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 329310 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 329311 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 329312 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 329313 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 329314 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 329315 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 329322 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 329325 INFO  (zkConnectionManagerCallback-1728-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 329325 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 329327 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]-SendThread(127.0.0.1:44881)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103af8026650002.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103af8026650002, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103af8026650002, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 329428 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testFullReplication
   [junit4]   2> 329602 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 329602 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 329602 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 329618 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 329618 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 329618 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 329619 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1dde92d5{/nt/b,null,AVAILABLE}
   [junit4]   2> 329620 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@3028fe49{SSL, (ssl, http/1.1)}{127.0.0.1:44501}
   [junit4]   2> 329620 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.Server Started @329647ms
   [junit4]   2> 329620 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard1, hostContext=/nt/b, hostPort=44501, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001/jetty-001/cores}
   [junit4]   2> 329620 ERROR (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 329620 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 329620 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.9.0
   [junit4]   2> 329620 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 329620 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 329620 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-03T14:52:00.104Z
   [junit4]   2> 329622 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 329624 INFO  (zkConnectionManagerCallback-1730-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 329624 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 329626 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]-SendThread(127.0.0.1:44881)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103af8026650003.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103af8026650003, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103af8026650003, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 329727 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 329727 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001/jetty-001/solr.xml
   [junit4]   2> 329730 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 329730 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 329731 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@14909ef6, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 330040 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 330040 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2
   [junit4]   2> 330041 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3db4ce5b[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 330041 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3db4ce5b[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 330043 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2
   [junit4]   2> 330044 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2a6aa1f2[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 330044 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2a6aa1f2[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 330045 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44881/solr
   [junit4]   2> 330049 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 330050 INFO  (zkConnectionManagerCallback-1741-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 330050 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 330051 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]-SendThread(127.0.0.1:44881)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103af8026650004.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103af8026650004, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103af8026650004, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 330154 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 330155 INFO  (zkConnectionManagerCallback-1743-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 330155 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 330268 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44501_nt%2Fb
   [junit4]   2> 330269 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.Overseer Overseer (id=73094983902887941-127.0.0.1:44501_nt%2Fb-n_0000000000) starting
   [junit4]   2> 330278 INFO  (OverseerStateUpdate-73094983902887941-127.0.0.1:44501_nt%2Fb-n_0000000000) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:44501_nt%2Fb
   [junit4]   2> 330287 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44501_nt%2Fb
   [junit4]   2> 330288 INFO  (zkCallback-1742-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 330290 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 330290 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 330316 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 330338 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14909ef6
   [junit4]   2> 330345 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14909ef6
   [junit4]   2> 330345 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14909ef6
   [junit4]   2> 330346 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001/jetty-001/cores
   [junit4]   2> 330571 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 330571 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 330571 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 330572 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 330572 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 330572 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 330572 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f3d899e{/nt/b,null,AVAILABLE}
   [junit4]   2> 330573 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@572386f7{SSL, (ssl, http/1.1)}{127.0.0.1:32787}
   [junit4]   2> 330573 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.Server Started @330600ms
   [junit4]   2> 330573 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {shards=shard2, hostContext=/nt/b, hostPort=32787, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001/jetty-002/cores}
   [junit4]   2> 330573 ERROR (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 330574 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 330574 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 8.9.0
   [junit4]   2> 330574 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 330574 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 330574 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-03T14:52:01.058Z
   [junit4]   2> 330578 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 330578 INFO  (zkConnectionManagerCallback-1757-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 330578 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 330681 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 330681 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001/jetty-002/solr.xml
   [junit4]   2> 330684 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 330684 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 330685 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@14909ef6, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 331482 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 331483 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2
   [junit4]   2> 331484 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@1b69111e[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 331484 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@1b69111e[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 331490 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.i.Http2SolrClient Create Http2SolrClient with HTTP/1.1 transport since Java 8 or lower versions does not support SSL + HTTP/2
   [junit4]   2> 331491 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@1a8e15aa[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 331491 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@1a8e15aa[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 331492 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44881/solr
   [junit4]   2> 331494 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 331495 INFO  (zkConnectionManagerCallback-1768-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 331495 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 331496 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]-SendThread(127.0.0.1:44881)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103af8026650007.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103af8026650007, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103af8026650007, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 331597 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 331599 INFO  (zkConnectionManagerCallback-1770-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 331599 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 331603 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 331608 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.ZkController Publish node=127.0.0.1:32787_nt%2Fb as DOWN
   [junit4]   2> 331609 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 331609 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32787_nt%2Fb
   [junit4]   2> 331612 INFO  (zkCallback-1742-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 331612 INFO  (zkCallback-1769-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 331614 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 331614 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 331637 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 331658 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14909ef6
   [junit4]   2> 331669 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14909ef6
   [junit4]   2> 331669 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14909ef6
   [junit4]   2> 331670 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001/jetty-002/cores
   [junit4]   2> 331688 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 331690 INFO  (zkConnectionManagerCallback-1783-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 331690 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 331691 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 331693 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44881/solr ready
   [junit4]   2> 331724 INFO  (OverseerThreadFactory-1750-thread-1-processing-n:127.0.0.1:44501_nt%2Fb) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.a.c.CreateCollectionCmd Create collection tmp_collection
   [junit4]   2> 331829 WARN  (OverseerThreadFactory-1750-thread-1-processing-n:127.0.0.1:44501_nt%2Fb) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (tmp_collection) without cores.
   [junit4]   2> 331839 INFO  (qtp143730251-3081) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 331844 INFO  (qtp143730251-3081) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=tmp_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&wt=javabin&version=2} status=0 QTime=123
   [junit4]   2> 331848 INFO  (OverseerCollectionConfigSetProcessor-73094983902887941-127.0.0.1:44501_nt%2Fb-n_0000000000) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 331850 INFO  (OverseerThreadFactory-1750-thread-2-processing-n:127.0.0.1:44501_nt%2Fb) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:44501_nt%2Fb for creating new replica of shard shard1 for collection tmp_collection
   [junit4]   2> 331853 INFO  (OverseerThreadFactory-1750-thread-2-processing-n:127.0.0.1:44501_nt%2Fb) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 331887 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb    x:tmp_collection_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=tmp_collection_shard1_replica_n1&action=CREATE&collection=tmp_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 331887 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb    x:tmp_collection_shard1_replica_n1 ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 332910 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 332921 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 332925 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 332969 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'tmp_collection_shard1_replica_n1' using configuration from configset conf1, trusted=true
   [junit4]   2> 332970 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.tmp_collection.shard1.replica_n1' (registry 'solr.core.tmp_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14909ef6
   [junit4]   2> 332971 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [[tmp_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001/jetty-001/cores/tmp_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001/jetty-001/cores/tmp_collection_shard1_replica_n1/data/]
   [junit4]   2> 332974 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=27, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=2.0185546875, noCFSRatio=0.0]
   [junit4]   2> 333058 WARN  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.D.s.CdcrRequestHandler Solr loaded a deprecated plugin/analysis class [solr.CdcrRequestHandler]. Please consult documentation how to replace it accordingly.
   [junit4]   2> 333058 WARN  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.h.CdcrRequestHandler CDCR (in its current form) is deprecated as of 8.6 and shall be removed in 9.0. See SOLR-14022 for details.
   [junit4]   2> 333077 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 333077 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 333079 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 333079 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 333080 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=25, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 333082 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 333083 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 333083 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 333087 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.h.CdcrBufferStateManager Created znode /collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 333091 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.h.CdcrProcessStateManager Created znode /collections/tmp_collection/cdcr/state/process
   [junit4]   2> 333102 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693222828597313536
   [junit4]   2> 333105 INFO  (searcherExecutor-1785-thread-1-processing-n:127.0.0.1:44501_nt%2Fb x:tmp_collection_shard1_replica_n1 c:tmp_collection s:shard1) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [tmp_collection_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 333109 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tmp_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 333109 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/tmp_collection/leaders/shard1
   [junit4]   2> 333113 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 333113 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 333113 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:44501/nt/b/tmp_collection_shard1_replica_n1/
   [junit4]   2> 333113 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 333114 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy https://127.0.0.1:44501/nt/b/tmp_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 333114 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/tmp_collection/leaders/shard1/leader after winning as /collections/tmp_collection/leader_elect/shard1/election/73094983902887941-core_node2-n_0000000000
   [junit4]   2> 333114 INFO  (zkCallback-1742-thread-1) [     ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ tmp_collection:shard1
   [junit4]   2> 333118 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:44501/nt/b/tmp_collection_shard1_replica_n1/ shard1
   [junit4]   2> 333219 INFO  (zkCallback-1742-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 333220 INFO  (zkCallback-1742-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 333220 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 333225 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=tmp_collection_shard1_replica_n1&action=CREATE&collection=tmp_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1338
   [junit4]   2> 333227 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:44501_nt%252Fb&action=ADDREPLICA&collection=tmp_collection&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1382
   [junit4]   2> 333231 INFO  (OverseerCollectionConfigSetProcessor-73094983902887941-127.0.0.1:44501_nt%2Fb-n_0000000000) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 333231 INFO  (OverseerThreadFactory-1750-thread-3-processing-n:127.0.0.1:44501_nt%2Fb) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:32787_nt%2Fb for creating new replica of shard shard1 for collection tmp_collection
   [junit4]   2> 333232 INFO  (OverseerThreadFactory-1750-thread-3-processing-n:127.0.0.1:44501_nt%2Fb) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 333268 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb    x:tmp_collection_shard1_replica_n3 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=tmp_collection_shard1_replica_n3&action=CREATE&collection=tmp_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 333374 INFO  (zkCallback-1742-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 333374 INFO  (zkCallback-1742-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 333374 INFO  (zkCallback-1742-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 334281 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 334291 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 334295 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 334313 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.c.CoreContainer Creating SolrCore 'tmp_collection_shard1_replica_n3' using configuration from configset conf1, trusted=true
   [junit4]   2> 334314 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.tmp_collection.shard1.replica_n3' (registry 'solr.core.tmp_collection.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@14909ef6
   [junit4]   2> 334314 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.c.SolrCore [[tmp_collection_shard1_replica_n3] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001/jetty-002/cores/tmp_collection_shard1_replica_n3], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001/jetty-002/cores/tmp_collection_shard1_replica_n3/data/]
   [junit4]   2> 334317 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=27, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=2.0185546875, noCFSRatio=0.0]
   [junit4]   2> 334366 WARN  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.h.CdcrRequestHandler CDCR (in its current form) is deprecated as of 8.6 and shall be removed in 9.0. See SOLR-14022 for details.
   [junit4]   2> 334367 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 334367 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 334368 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 334368 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 334369 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=25, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 334372 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 334373 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 334373 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 334375 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693222829932150784
   [junit4]   2> 334379 INFO  (searcherExecutor-1792-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.c.SolrCore [tmp_collection_shard1_replica_n3]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 334394 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tmp_collection/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=1}
   [junit4]   2> 334394 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/tmp_collection/leaders/shard1
   [junit4]   2> 334408 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1 r:core_node2 x:tmp_collection_shard1_replica_n1 ] o.a.s.c.S.Request [tmp_collection_shard1_replica_n1]  webapp=/nt/b path=/cdcr params={action=LASTPROCESSEDVERSION&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 334413 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1  x:tmp_collection_shard1_replica_n3 ] o.a.s.c.ZkController Core needs to recover:tmp_collection_shard1_replica_n3
   [junit4]   2> 334417 INFO  (updateExecutor-1764-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.u.DefaultSolrCoreState Running recovery
   [junit4]   2> 334422 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
   [junit4]   2> 334423 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy startupVersions is empty
   [junit4]   2> 334423 INFO  (qtp143730251-3080) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=tmp_collection_shard1_replica_n3&action=CREATE&collection=tmp_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1155
   [junit4]   2> 334430 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:32787_nt%252Fb&action=ADDREPLICA&collection=tmp_collection&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1202
   [junit4]   2> 334435 WARN  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]-SendThread(127.0.0.1:44881)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103af8026650009.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103af8026650009, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103af8026650009, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 334457 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1 r:core_node2 x:tmp_collection_shard1_replica_n1 ] o.a.s.c.S.Request [tmp_collection_shard1_replica_n1]  webapp=/nt/b path=/admin/ping params={wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 334457 INFO  (qtp937822663-3043) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1 r:core_node2 x:tmp_collection_shard1_replica_n1 ] o.a.s.c.S.Request [tmp_collection_shard1_replica_n1]  webapp=/nt/b path=/admin/ping params={wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 334458 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[tmp_collection_shard1_replica_n3]
   [junit4]   2> 334458 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 334458 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Publishing state of core [tmp_collection_shard1_replica_n3] as recovering, leader is [https://127.0.0.1:44501/nt/b/tmp_collection_shard1_replica_n1/] and I am [https://127.0.0.1:32787/nt/b/tmp_collection_shard1_replica_n3/]
   [junit4]   2> 334460 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:44501/nt/b]; [WaitForState: action=PREPRECOVERY&core=tmp_collection_shard1_replica_n1&nodeName=127.0.0.1:32787_nt%252Fb&coreNodeName=core_node4&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
   [junit4]   2> 334462 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb    x:tmp_collection_shard1_replica_n1 ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node4, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
   [junit4]   2> 334463 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=tmp_collection, shard=shard1, thisCore=tmp_collection_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:32787_nt%2Fb, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{
   [junit4]   2>   "core":"tmp_collection_shard1_replica_n3",
   [junit4]   2>   "node_name":"127.0.0.1:32787_nt%2Fb",
   [junit4]   2>   "base_url":"https://127.0.0.1:32787/nt/b",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 334463 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=tmp_collection, shard=shard1, thisCore=tmp_collection_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:32787_nt%2Fb, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{
   [junit4]   2>   "core":"tmp_collection_shard1_replica_n3",
   [junit4]   2>   "node_name":"127.0.0.1:32787_nt%2Fb",
   [junit4]   2>   "base_url":"https://127.0.0.1:32787/nt/b",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 334463 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=tmp_collection, shard=shard1, thisCore=tmp_collection_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:32787_nt%2Fb, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{
   [junit4]   2>   "core":"tmp_collection_shard1_replica_n3",
   [junit4]   2>   "node_name":"127.0.0.1:32787_nt%2Fb",
   [junit4]   2>   "base_url":"https://127.0.0.1:32787/nt/b",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "type":"NRT"}
   [junit4]   2> 334536 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: tmp_collection failOnTimeout: true timeout (sec):
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:down live:true
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:down live:true
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:down live:true
   [junit4]   2> 334564 INFO  (zkCallback-1742-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 334564 INFO  (zkCallback-1742-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 334564 INFO  (zkCallback-1769-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 334564 INFO  (zkCallback-1742-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 334564 INFO  (zkCallback-1769-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 334564 INFO  (zkCallback-1769-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:recovering live:true
   [junit4]   2> 334565 INFO  (watches-1744-thread-2) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=tmp_collection, shard=shard1, thisCore=tmp_collection_shard1_replica_n1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:32787_nt%2Fb, coreNodeName=core_node4, onlyIfActiveCheckResult=false, nodeProps: core_node4:{
   [junit4]   2>   "core":"tmp_collection_shard1_replica_n3",
   [junit4]   2>   "node_name":"127.0.0.1:32787_nt%2Fb",
   [junit4]   2>   "base_url":"https://127.0.0.1:32787/nt/b",
   [junit4]   2>   "state":"recovering",
   [junit4]   2>   "type":"NRT"}
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:recovering live:true
   [junit4]   2> 334565 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:32787_nt%252Fb&onlyIfLeaderActive=true&core=tmp_collection_shard1_replica_n1&coreNodeName=core_node4&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=103
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:recovering live:true
   [junit4]   2> 335066 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [https://127.0.0.1:44501/nt/b/tmp_collection_shard1_replica_n1/] - recoveringAfterStartup=[true]
   [junit4]   2> 335066 WARN  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.u.PeerSyncWithLeader no frame of reference to tell if we've missed updates
   [junit4]   2> 335066 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy PeerSync Recovery was not successful - trying replication.
   [junit4]   2> 335066 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Starting Replication Recovery.
   [junit4]   2> 335066 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Attempting to replicate from [https://127.0.0.1:44501/nt/b/tmp_collection_shard1_replica_n1/].
   [junit4]   2> 335110 INFO  (qtp143730251-3076) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.u.p.DistributedUpdateProcessor Ignoring commit while not ACTIVE - state: BUFFERING replay: false
   [junit4]   2> 335111 INFO  (qtp143730251-3076) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.S.Request [tmp_collection_shard1_replica_n3]  webapp=/nt/b path=/update params={update.distrib=FROMLEADER&update.chain=cdcr-processor-chain&waitSearcher=true&openSearcher=false&commit=true&softCommit=false&distrib.from=https://127.0.0.1:44501/nt/b/tmp_collection_shard1_replica_n1/&commit_end_point=replicas&wt=javabin&version=2&expungeDeletes=false} status=0 QTime=0
   [junit4]   2> 335112 INFO  (qtp937822663-3041) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1 r:core_node2 x:tmp_collection_shard1_replica_n1 ] o.a.s.c.S.Request [tmp_collection_shard1_replica_n1]  webapp=/nt/b path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=45
   [junit4]   2> 335113 INFO  (qtp937822663-3042) [n:127.0.0.1:44501_nt%2Fb c:tmp_collection s:shard1 r:core_node2 x:tmp_collection_shard1_replica_n1 ] o.a.s.c.S.Request [tmp_collection_shard1_replica_n1]  webapp=/nt/b path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 335114 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.h.IndexFetcher Leader's generation: 1
   [junit4]   2> 335114 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.h.IndexFetcher Leader's version: 0
   [junit4]   2> 335114 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.h.IndexFetcher Follower's generation: 1
   [junit4]   2> 335114 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.h.IndexFetcher Follower's version: 0
   [junit4]   2> 335114 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.h.IndexFetcher New index in Leader. Deleting mine...
   [junit4]   2> 335117 INFO  (searcherExecutor-1792-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.SolrCore [tmp_collection_shard1_replica_n3]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 335117 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy No replay needed.
   [junit4]   2> 335117 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Replication Recovery was successful.
   [junit4]   2> 335117 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
   [junit4]   2> 335118 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Updating version bucket highest from index after successful recovery.
   [junit4]   2> 335118 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693222830711242752
   [junit4]   2> 335119 INFO  (recoveryExecutor-1766-thread-1-processing-n:127.0.0.1:32787_nt%2Fb x:tmp_collection_shard1_replica_n3 c:tmp_collection s:shard1 r:core_node4) [n:127.0.0.1:32787_nt%2Fb c:tmp_collection s:shard1 r:core_node4 x:tmp_collection_shard1_replica_n3 ] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[true]
   [junit4]   2> 335223 INFO  (zkCallback-1742-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 335223 INFO  (zkCallback-1742-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 335223 INFO  (zkCallback-1769-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 335223 INFO  (zkCallback-1769-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 335223 INFO  (zkCallback-1742-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 335223 INFO  (zkCallback-1769-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   1> replica:core_node2 rstate:active live:true
   [junit4]   1> replica:core_node4 rstate:active live:true
   [junit4]   1> no one is recoverying
   [junit4]   2> 335223 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: tmp_collection
   [junit4]   2> 335225 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 335226 INFO  (zkConnectionManagerCallback-1803-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 335226 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 335227 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 335229 INFO  (TEST-CdcrReplicationHandlerTest.testFullReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44881/solr ready
   [junit4]   2> 335232 INFO  (OverseerCollectionConfigSetProcessor-73094983902887941-127.0.0.1:44501_nt%2Fb-n_0000000000) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 335267 INFO  (OverseerThreadFactory-1750-thread-4-processing-n:127.0.0.1:44501_nt%2Fb) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true&deleteMetricsHistory=true, asyncId=null
   [junit4]   2> 335275 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb    x:tmp_collection_shard1_replica_n1 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tmp_collection.shard1.replica_n1 tag=null
   [junit4]   2> 335275 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb    x:tmp_collection_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7de1c3ac: rootName = null, domain = solr.core.tmp_collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.tmp_collection.shard1.replica_n1/com.codahale.metrics.MetricRegistry@333e1a07
   [junit4]   2> 335276 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb    x:tmp_collection_shard1_replica_n3 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tmp_collection.shard1.replica_n3 tag=null
   [junit4]   2> 335276 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb    x:tmp_collection_shard1_replica_n3 ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2e743d10: rootName = null, domain = solr.core.tmp_collection.shard1.replica_n3, service url = null, agent id = null] for registry solr.core.tmp_collection.shard1.replica_n3/com.codahale.metrics.MetricRegistry@4bbeede3
   [junit4]   2> 335308 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.SolrCore [tmp_collection_shard1_replica_n3]  CLOSING SolrCore org.apache.solr.core.SolrCore@56bc396
   [junit4]   2> 335308 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tmp_collection.shard1.replica_n3 tag=SolrCore@56bc396
   [junit4]   2> 335308 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.tmp_collection.shard1.leader tag=SolrCore@56bc396
   [junit4]   2> 335308 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR handler @ tmp_collection:shard1
   [junit4]   2> 335308 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 335312 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.SolrCore [tmp_collection_shard1_replica_n1]  CLOSING SolrCore org.apache.solr.core.SolrCore@27b1ab55
   [junit4]   2> 335312 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tmp_collection.shard1.replica_n1 tag=SolrCore@27b1ab55
   [junit4]   2> 335312 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.tmp_collection.shard1.leader tag=SolrCore@27b1ab55
   [junit4]   2> 335312 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR handler @ tmp_collection:shard1
   [junit4]   2> 335313 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 335323 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tmp_collection/terms/shard1 to Terms{values={core_node4=0}, version=2}
   [junit4]   2> 335324 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
   [junit4]   2> 335325 INFO  (qtp937822663-3045) [n:127.0.0.1:44501_nt%2Fb     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&deleteMetricsHistory=true&core=tmp_collection_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=50
   [junit4]   2> 335326 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tmp_collection/terms/shard1 to Terms{values={}, version=3}
   [junit4]   2> 335327 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.c.ShardLeaderElectionContextBase No version found for ephemeral leader parent node, won't remove previous leader registration.
   [junit4]   2> 335327 INFO  (qtp143730251-3078) [n:127.0.0.1:32787_nt%2Fb     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&deleteMetricsHistory=true&core=tmp_collection_shard1_replica_n3&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=51
   [junit4]   2> 335432 INFO  (zkCallback-1742-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/tmp_collection/state.json] for collection [tmp_collection] has occurred - updating... (live

[...truncated too long message...]

dcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@64c3cd3f: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@41058750
   [junit4]   2> 442151 INFO  (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
   [junit4]   2> 442153 INFO  (closeThreadPool-2821-thread-2) [     ] o.a.s.c.Overseer Overseer (id=73094990269841416-127.0.0.1:34847_nt%2Fb-n_0000000001) closing
   [junit4]   2> 442154 INFO  (OverseerAutoScalingTriggerThread-73094990269841416-127.0.0.1:34847_nt%2Fb-n_0000000001) [     ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 442156 INFO  (OverseerStateUpdate-73094990269841416-127.0.0.1:34847_nt%2Fb-n_0000000001) [n:127.0.0.1:34847_nt%2Fb     ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34847_nt%2Fb
   [junit4]   2> 442160 INFO  (closeThreadPool-2821-thread-1) [     ] o.a.s.c.Overseer Overseer (id=73094990269841416-127.0.0.1:34847_nt%2Fb-n_0000000001) closing
   [junit4]   2> 442260 INFO  (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.Overseer Overseer (id=73094990269841416-127.0.0.1:34847_nt%2Fb-n_0000000001) closing
   [junit4]   2> 442261 INFO  (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@590aaf5c{SSL, (ssl, http/1.1)}{127.0.0.1:0}
   [junit4]   2> 442261 INFO  (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@17f62e65{/nt/b,null,STOPPED}
   [junit4]   2> 442261 INFO  (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 442262 INFO  (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testPartialReplication
   [junit4]   2> 442262 INFO  (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 442364 WARN  (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]-SendThread(127.0.0.1:41427)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103af81a1e50001.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103af81a1e50001, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103af81a1e50001, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 442466 WARN  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	22	/solr/aliases.json
   [junit4]   2> 	14	/solr/collections/source_collection/terms/shard1
   [junit4]   2> 	8	/solr/collections/tmp_collection/state.json
   [junit4]   2> 	8	/solr/collections/tmp_collection/terms/shard1
   [junit4]   2> 	6	/solr/collections/source_collection/leaders/shard1/leader
   [junit4]   2> 	5	/solr/configs/conf1
   [junit4]   2> 	4	/solr/collections/tmp_collection/leaders/shard1/leader
   [junit4]   2> 	3	/solr/packages.json
   [junit4]   2> 	3	/solr/security.json
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	46	/solr/collections/source_collection/state.json
   [junit4]   2> 	24	/solr/collections/tmp_collection/state.json
   [junit4]   2> 	22	/solr/clusterprops.json
   [junit4]   2> 	22	/solr/clusterstate.json
   [junit4]   2> 	4	/solr/collections/source_collection/cdcr/state/buffer
   [junit4]   2> 	4	/solr/collections/source_collection/cdcr/state/process
   [junit4]   2> 	2	/solr/collections/tmp_collection/cdcr/state/buffer
   [junit4]   2> 	2	/solr/collections/source_collection/leader_elect/shard1/election/73094990269841416-core_node4-n_0000000000
   [junit4]   2> 	2	/solr/autoscaling.json
   [junit4]   2> 	2	/solr/collections/tmp_collection/cdcr/state/process
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	45	/solr/overseer/queue
   [junit4]   2> 	31	/solr/collections
   [junit4]   2> 	29	/solr/live_nodes
   [junit4]   2> 	21	/solr/overseer/collection-queue-work
   [junit4]   2> 	11	/solr/collections/source_collection/state.json
   [junit4]   2> 	7	/solr/collections/tmp_collection/state.json
   [junit4]   2> 	2	/solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 	2	/solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 	2	/solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> 442470 INFO  (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:41427
   [junit4]   2> 442470 INFO  (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:41427
   [junit4]   2> 442470 INFO  (TEST-CdcrReplicationHandlerTest.testPartialReplication-seed#[C84D940E8CA4D6FC]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 41427
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J3/temp/solr.cloud.cdcr.CdcrReplicationHandlerTest_C84D940E8CA4D6FC-001
   [junit4]   2> Mar 03, 2021 2:53:53 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene87): {_root_=Lucene84, id=BlockTreeOrds(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=1853, maxMBSortInHeap=6.687768866191895, sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=es-US, timezone=Asia/Bangkok
   [junit4]   2> NOTE: Linux 5.4.0-45-generic amd64/Oracle Corporation 1.8.0_281 (64-bit)/cpus=4,threads=1,free=186797208,total=515899392
   [junit4]   2> NOTE: All tests run in this JVM: [SecurityConfHandlerTest, CollectionPropsTest, TestJsonRangeFacets, TestConfigSetsAPIExclusivity, SplitHandlerTest, ClassificationUpdateProcessorIntegrationTest, UpdateLogTest, TestCloudPhrasesIdentificationComponent, TestSolrDeletionPolicy2, MetricsConfigTest, CSVRequestHandlerTest, SmileWriterTest, TestClassicSimilarityFactory, TestLMDirichletSimilarityFactory, SuggesterTest, PeerSyncWithIndexFingerprintCachingTest, TestDocTermOrdsUninvertLimit, OutOfBoxZkACLAndCredentialsProvidersTest, HdfsRestartWhileUpdatingTest, BigEndianAscendingWordSerializerTest, BJQParserTest, DocValuesMultiTest, TestManagedResourceStorage, NodeAddedTriggerIntegrationTest, TestNumericRangeQuery64, SolrLogAuditLoggerPluginTest, TestRandomFlRTGCloud, BasicAuthOnSingleNodeTest, TestPrepRecovery, CdcrReplicationHandlerTest]
   [junit4] Completed [95/934 (1!)] on J3 in 114.01s, 5 tests, 1 error <<< FAILURES!

[...truncated 53561 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:652: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:507: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:494: Source checkout is dirty (unversioned/missing files) after running tests!!! Offending files:
* solr/contrib/analytics/src/test-files/solr/filestore
* solr/contrib/analytics/src/test-files/solr/userfiles
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/collection1/conf/dataimport.properties
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/filestore
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/userfiles
* solr/contrib/dataimporthandler/src/test-files/dih/solr/filestore
* solr/contrib/dataimporthandler/src/test-files/dih/solr/userfiles
* solr/contrib/extraction/src/test-files/extraction/solr/filestore
* solr/contrib/extraction/src/test-files/extraction/solr/userfiles
* solr/contrib/langid/src/test-files/langid/solr/filestore
* solr/contrib/langid/src/test-files/langid/solr/userfiles
* solr/contrib/ltr/src/test-files/solr/filestore
* solr/contrib/ltr/src/test-files/solr/userfiles
* solr/contrib/prometheus-exporter/src/test-files/solr/filestore
* solr/contrib/prometheus-exporter/src/test-files/solr/userfiles
* solr/contrib/velocity/src/test-files/velocity/solr/filestore
* solr/contrib/velocity/src/test-files/velocity/solr/userfiles
* solr/core/src/test-files/solr/filestore
* solr/core/src/test-files/solr/userfiles
* solr/solrj/src/test-files/solrj/solr/multicore/filestore
* solr/solrj/src/test-files/solrj/solr/multicore/userfiles

Total time: 286 minutes 17 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene » Lucene-Solr-NightlyTests-8.x - Build # 198 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-NightlyTests-8.x/198/

2 tests failed.
FAILED:  org.apache.solr.metrics.reporters.solr.SolrCloudReportersTest.testDefaultPlugins

Error Message:
Timeout occurred while waiting response from server at: http://127.0.0.1:37169/solr

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occurred while waiting response from server at: http://127.0.0.1:37169/solr
	at __randomizedtesting.SeedInfo.seed([94FC445D9EA98321:DC7C168DC2E9A00C]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:692)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:266)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:369)
	at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:297)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1171)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:934)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:866)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:214)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:231)
	at org.apache.solr.metrics.reporters.solr.SolrCloudReportersTest.testDefaultPlugins(SolrCloudReportersTest.java:176)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:571)
	... 53 more


FAILED:  junit.framework.TestSuite.org.apache.solr.response.transform.TestSubQueryTransformerDistrib

Error Message:
Timeout occurred while waiting response from server at: http://127.0.0.1:33487/solr

Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occurred while waiting response from server at: http://127.0.0.1:33487/solr
	at __randomizedtesting.SeedInfo.seed([94FC445D9EA98321]:0)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:692)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:266)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
	at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:369)
	at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:297)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1171)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:934)
	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:866)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:214)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:231)
	at org.apache.solr.response.transform.TestSubQueryTransformerDistrib.setupCluster(TestSubQueryTransformerDistrib.java:83)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:878)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
	at java.net.SocketInputStream.read(SocketInputStream.java:171)
	at java.net.SocketInputStream.read(SocketInputStream.java:141)
	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:571)
	... 36 more




Build Log:
[...truncated 13554 lines...]
   [junit4] Suite: org.apache.solr.metrics.reporters.solr.SolrCloudReportersTest
   [junit4]   2> 379917 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 379918 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/data-dir-21-001
   [junit4]   2> 379918 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 379919 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
   [junit4]   2> 379919 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 379921 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.SolrCloudTestCase Using per-replica state
   [junit4]   2> 379921 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 0 servers in /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-001
   [junit4]   2> 379921 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 379921 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 379922 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 379926 WARN  (ZkTestServer Run Thread) [     ] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0.
   [junit4]   2> 380021 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.ZkTestServer start zk server on port: 43389
   [junit4]   2> 380022 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:43389
   [junit4]   2> 380022 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:43389
   [junit4]   2> 380022 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 43389
   [junit4]   2> 380035 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 380055 INFO  (zkConnectionManagerCallback-2570-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 380055 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 380094 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 380111 INFO  (zkConnectionManagerCallback-2572-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 380111 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 380115 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 380122 INFO  (zkConnectionManagerCallback-2574-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 380122 INFO  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 380127 WARN  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker-SendThread(127.0.0.1:43389)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103aa7069230002.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103aa7069230002, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103aa7069230002, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 380237 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testDefaultPlugins
   [junit4]   2> 380238 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
   [junit4]   2> 380239 WARN  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker-SendThread(127.0.0.1:43389)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103aa7069230000.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103aa7069230000, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103aa7069230000, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 380485 WARN  (SUITE-SolrCloudReportersTest-seed#[94FC445D9EA98321]-worker-SendThread(127.0.0.1:43389)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103aa7069230001.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103aa7069230001, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103aa7069230001, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 380587 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:43389
   [junit4]   2> 380587 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:43389
   [junit4]   2> 380587 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 43389
   [junit4]   2> 380588 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-002
   [junit4]   2> 380588 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 380588 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 380589 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 380589 WARN  (ZkTestServer Run Thread) [     ] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0.
   [junit4]   2> 380688 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.ZkTestServer start zk server on port: 34879
   [junit4]   2> 380689 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:34879
   [junit4]   2> 380689 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:34879
   [junit4]   2> 380689 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 34879
   [junit4]   2> 380703 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 380717 INFO  (zkConnectionManagerCallback-2579-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 380718 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 380732 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 380733 INFO  (zkConnectionManagerCallback-2581-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 380733 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 380735 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 380740 INFO  (zkConnectionManagerCallback-2583-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 380740 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 380741 WARN  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]-SendThread(127.0.0.1:34879)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103aa706bb70002.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103aa706bb70002, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103aa706bb70002, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 380844 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 380844 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 380844 INFO  (jetty-launcher-2584-thread-1) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 380848 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 380848 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 380848 INFO  (jetty-launcher-2584-thread-2) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 380876 INFO  (jetty-launcher-2584-thread-1) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 380876 INFO  (jetty-launcher-2584-thread-1) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 380876 INFO  (jetty-launcher-2584-thread-1) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 380879 INFO  (jetty-launcher-2584-thread-1) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@54295c37{/solr,null,AVAILABLE}
   [junit4]   2> 380892 INFO  (jetty-launcher-2584-thread-2) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 380892 INFO  (jetty-launcher-2584-thread-2) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 380892 INFO  (jetty-launcher-2584-thread-2) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 380901 INFO  (jetty-launcher-2584-thread-2) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@150e852f{/solr,null,AVAILABLE}
   [junit4]   2> 380902 INFO  (jetty-launcher-2584-thread-2) [     ] o.e.j.s.AbstractConnector Started ServerConnector@3209611{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:37169}
   [junit4]   2> 380902 INFO  (jetty-launcher-2584-thread-2) [     ] o.e.j.s.Server Started @380931ms
   [junit4]   2> 380902 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, zkHost=127.0.0.1:34879/solr, hostPort=37169}
   [junit4]   2> 380904 ERROR (jetty-launcher-2584-thread-2) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 380904 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 380904 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 380904 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 380904 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 380904 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-02T15:16:43.063Z
   [junit4]   2> 380939 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 381034 INFO  (zkConnectionManagerCallback-2586-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 381034 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 381035 INFO  (jetty-launcher-2584-thread-1) [     ] o.e.j.s.AbstractConnector Started ServerConnector@6db97e27{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:40507}
   [junit4]   2> 381036 INFO  (jetty-launcher-2584-thread-1) [     ] o.e.j.s.Server Started @381064ms
   [junit4]   2> 381036 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, zkHost=127.0.0.1:34879/solr, hostPort=40507}
   [junit4]   2> 381036 ERROR (jetty-launcher-2584-thread-1) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 381036 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 381036 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 381036 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 381036 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 381036 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-02T15:16:43.195Z
   [junit4]   2> 381036 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 381037 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 381039 INFO  (zkConnectionManagerCallback-2588-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 381039 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 381039 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 381041 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 381041 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 381041 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 381041 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 381042 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 381043 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 381044 WARN  (jetty-launcher-2584-thread-1-SendThread(127.0.0.1:34879)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103aa706bb70004.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103aa706bb70004, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103aa706bb70004, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 382453 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 382454 WARN  (jetty-launcher-2584-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@639a9632[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 382454 WARN  (jetty-launcher-2584-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@639a9632[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 382457 WARN  (jetty-launcher-2584-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@ea67df7[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 382457 WARN  (jetty-launcher-2584-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@ea67df7[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 382458 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34879/solr
   [junit4]   2> 382459 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 382460 INFO  (zkConnectionManagerCallback-2602-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 382460 INFO  (jetty-launcher-2584-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 382562 INFO  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 382563 INFO  (zkConnectionManagerCallback-2604-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 382563 INFO  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 382671 INFO  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37169_solr
   [junit4]   2> 382672 INFO  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.c.Overseer Overseer (id=73089418788274182-127.0.0.1:37169_solr-n_0000000000) starting
   [junit4]   2> 382691 INFO  (OverseerStateUpdate-73089418788274182-127.0.0.1:37169_solr-n_0000000000) [n:127.0.0.1:37169_solr     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:37169_solr
   [junit4]   2> 382691 INFO  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37169_solr
   [junit4]   2> 382694 INFO  (OverseerStateUpdate-73089418788274182-127.0.0.1:37169_solr-n_0000000000) [n:127.0.0.1:37169_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 382695 INFO  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 382696 WARN  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 382711 INFO  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 382735 INFO  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527
   [junit4]   2> 382744 INFO  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527
   [junit4]   2> 382745 INFO  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527
   [junit4]   2> 382746 INFO  (jetty-launcher-2584-thread-2) [n:127.0.0.1:37169_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-002/node2
   [junit4]   2> 384096 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
   [junit4]   2> 384097 WARN  (jetty-launcher-2584-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@7784001a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 384097 WARN  (jetty-launcher-2584-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@7784001a[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 384102 WARN  (jetty-launcher-2584-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@7693cd7f[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 384102 WARN  (jetty-launcher-2584-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@7693cd7f[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 384103 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34879/solr
   [junit4]   2> 384107 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 384111 INFO  (zkConnectionManagerCallback-2624-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 384112 INFO  (jetty-launcher-2584-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 384214 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 384215 INFO  (zkConnectionManagerCallback-2626-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 384215 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 384220 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 384223 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.c.ZkController Publish node=127.0.0.1:40507_solr as DOWN
   [junit4]   2> 384227 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 384227 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40507_solr
   [junit4]   2> 384232 INFO  (zkCallback-2603-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 384234 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 384234 WARN  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 384242 INFO  (zkCallback-2625-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 384279 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 384300 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527
   [junit4]   2> 384310 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527
   [junit4]   2> 384311 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527
   [junit4]   2> 384312 INFO  (jetty-launcher-2584-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-002/node1
   [junit4]   2> 384331 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=2
   [junit4]   2> 384336 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 384337 INFO  (zkConnectionManagerCallback-2639-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 384337 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 384339 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 384340 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34879/solr ready
   [junit4]   2> 384347 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 384347 INFO  (zkConnectionManagerCallback-2641-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 384348 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 384465 INFO  (OverseerThreadFactory-2611-thread-1-processing-n:127.0.0.1:37169_solr) [n:127.0.0.1:37169_solr     ] o.a.s.c.a.c.CreateCollectionCmd Create collection test_collection
   [junit4]   2> 384494 INFO  (OverseerThreadFactory-2611-thread-1-processing-n:127.0.0.1:37169_solr) [n:127.0.0.1:37169_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "core":"test_collection_shard1_replica_n1",
   [junit4]   2>   "node_name":"127.0.0.1:37169_solr",
   [junit4]   2>   "base_url":"http://127.0.0.1:37169/solr",
   [junit4]   2>   "collection":"test_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 384517 INFO  (OverseerThreadFactory-2611-thread-1-processing-n:127.0.0.1:37169_solr) [n:127.0.0.1:37169_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "core":"test_collection_shard1_replica_n3",
   [junit4]   2>   "node_name":"127.0.0.1:40507_solr",
   [junit4]   2>   "base_url":"http://127.0.0.1:40507/solr",
   [junit4]   2>   "collection":"test_collection",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 384527 INFO  (OverseerThreadFactory-2611-thread-1-processing-n:127.0.0.1:37169_solr) [n:127.0.0.1:37169_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "core":"test_collection_shard2_replica_n5",
   [junit4]   2>   "node_name":"127.0.0.1:37169_solr",
   [junit4]   2>   "base_url":"http://127.0.0.1:37169/solr",
   [junit4]   2>   "collection":"test_collection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 384534 INFO  (OverseerThreadFactory-2611-thread-1-processing-n:127.0.0.1:37169_solr) [n:127.0.0.1:37169_solr     ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "core":"test_collection_shard2_replica_n7",
   [junit4]   2>   "node_name":"127.0.0.1:40507_solr",
   [junit4]   2>   "base_url":"http://127.0.0.1:40507/solr",
   [junit4]   2>   "collection":"test_collection",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 384548 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr    x:test_collection_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=test&newCollection=true&name=test_collection_shard1_replica_n1&action=CREATE&numShards=2&collection=test_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 384548 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr    x:test_collection_shard1_replica_n1 ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 384548 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr    x:test_collection_shard2_replica_n7 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=test&newCollection=true&name=test_collection_shard2_replica_n7&action=CREATE&numShards=2&collection=test_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 384549 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr    x:test_collection_shard2_replica_n5 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=test&newCollection=true&name=test_collection_shard2_replica_n5&action=CREATE&numShards=2&collection=test_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 384549 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr    x:test_collection_shard1_replica_n3 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=test&newCollection=true&name=test_collection_shard1_replica_n3&action=CREATE&numShards=2&collection=test_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 384575 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 384580 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 384584 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 384585 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 384585 WARN  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.s.IndexSchema no uniqueKey specified in schema.
   [junit4]   2> 384585 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field null
   [junit4]   2> 384586 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.c.CoreContainer Creating SolrCore 'test_collection_shard1_replica_n3' using configuration from configset test, trusted=true
   [junit4]   2> 384587 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.test_collection.shard1.replica_n3' (registry 'solr.core.test_collection.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527
   [junit4]   2> 384588 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.c.SolrCore [[test_collection_shard1_replica_n3] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-002/node1/test_collection_shard1_replica_n3], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-002/node1/test_collection_shard1_replica_n3/data/]
   [junit4]   2> 384593 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 384595 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 384595 WARN  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.s.IndexSchema no uniqueKey specified in schema.
   [junit4]   2> 384595 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field null
   [junit4]   2> 384595 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.CoreContainer Creating SolrCore 'test_collection_shard2_replica_n5' using configuration from configset test, trusted=true
   [junit4]   2> 384608 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 384608 WARN  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema no uniqueKey specified in schema.
   [junit4]   2> 384608 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field null
   [junit4]   2> 384608 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'test_collection_shard1_replica_n1' using configuration from configset test, trusted=true
   [junit4]   2> 384608 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.test_collection.shard2.replica_n5' (registry 'solr.core.test_collection.shard2.replica_n5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527
   [junit4]   2> 384608 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.SolrCore [[test_collection_shard2_replica_n5] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-002/node2/test_collection_shard2_replica_n5], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-002/node2/test_collection_shard2_replica_n5/data/]
   [junit4]   2> 384608 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.test_collection.shard1.replica_n1' (registry 'solr.core.test_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527
   [junit4]   2> 384608 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [[test_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-002/node2/test_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-002/node2/test_collection_shard1_replica_n1/data/]
   [junit4]   2> 384609 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.s.IndexSchema Schema name=minimal
   [junit4]   2> 384609 WARN  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.s.IndexSchema no uniqueKey specified in schema.
   [junit4]   2> 384610 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field null
   [junit4]   2> 384610 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.CoreContainer Creating SolrCore 'test_collection_shard2_replica_n7' using configuration from configset test, trusted=true
   [junit4]   2> 384614 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.test_collection.shard2.replica_n7' (registry 'solr.core.test_collection.shard2.replica_n7') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8757527
   [junit4]   2> 384614 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.SolrCore [[test_collection_shard2_replica_n7] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-002/node1/test_collection_shard2_replica_n7], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J1/temp/solr.metrics.reporters.solr.SolrCloudReportersTest_94FC445D9EA98321-001/tempDir-002/node1/test_collection_shard2_replica_n7/data/]
   [junit4]   2> 384875 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 384875 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 384882 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/test
   [junit4]   2> 384882 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/test
   [junit4]   2> 384883 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 384892 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 384892 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 384899 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/test
   [junit4]   2> 384899 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/test
   [junit4]   2> 384900 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 384903 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 384903 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 384911 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 384911 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 384917 INFO  (searcherExecutor-2647-thread-1-processing-n:127.0.0.1:37169_solr x:test_collection_shard1_replica_n1 c:test_collection s:shard1 r:core_node2) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [test_collection_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 384918 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/test
   [junit4]   2> 384918 INFO  (searcherExecutor-2645-thread-1-processing-n:127.0.0.1:37169_solr x:test_collection_shard2_replica_n5 c:test_collection s:shard2 r:core_node6) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.SolrCore [test_collection_shard2_replica_n5]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 384918 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/test
   [junit4]   2> 384919 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/test
   [junit4]   2> 384919 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 384923 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/test
   [junit4]   2> 384923 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 385233 INFO  (searcherExecutor-2643-thread-1-processing-n:127.0.0.1:40507_solr x:test_collection_shard1_replica_n3 c:test_collection s:shard1 r:core_node4) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.c.SolrCore [test_collection_shard1_replica_n3]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 385233 INFO  (searcherExecutor-2649-thread-1-processing-n:127.0.0.1:40507_solr x:test_collection_shard2_replica_n7 c:test_collection s:shard2 r:core_node8) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.SolrCore [test_collection_shard2_replica_n7]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 385236 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/test_collection/terms/shard2 to Terms{values={core_node6=0}, version=0}
   [junit4]   2> 385237 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/test_collection/leaders/shard2
   [junit4]   2> 385238 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/test_collection/terms/shard2 to Terms{values={core_node6=0, core_node8=0}, version=1}
   [junit4]   2> 385238 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/test_collection/leaders/shard2
   [junit4]   2> 385242 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
   [junit4]   2> 385242 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/test_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 385243 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/test_collection/leaders/shard1
   [junit4]   2> 385246 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/test_collection/terms/shard1 to Terms{values={core_node2=0, core_node4=0}, version=1}
   [junit4]   2> 385248 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 385248 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 385248 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40507/solr/test_collection_shard2_replica_n7/
   [junit4]   2> 385248 ERROR (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.SyncStrategy No UpdateLog found - cannot sync
   [junit4]   2> 385248 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 385248 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/test_collection/leaders/shard2/leader after winning as /collections/test_collection/leader_elect/shard2/election/73089418788274184-core_node8-n_0000000000
   [junit4]   2> 385250 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40507/solr/test_collection_shard2_replica_n7/ shard2
   [junit4]   2> 385257 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr c:test_collection s:shard1 r:core_node4 x:test_collection_shard1_replica_n3 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/test_collection/leaders/shard1
   [junit4]   2> 385258 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoute in=14992ms
   [junit4]   2> 385260 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr c:test_collection s:shard2 r:core_node8 x:test_collection_shard2_replica_n7 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 385441 INFO  (qtp1669237816-8216) [n:127.0.0.1:40507_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=test&newCollection=true&name=test_collection_shard2_replica_n7&action=CREATE&numShards=2&collection=test_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=892
   [junit4]   2> 385764 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 385764 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 385764 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37169/solr/test_collection_shard1_replica_n1/
   [junit4]   2> 385764 ERROR (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy No UpdateLog found - cannot sync
   [junit4]   2> 385764 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 385764 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/test_collection/leaders/shard1/leader after winning as /collections/test_collection/leader_elect/shard1/election/73089418788274182-core_node2-n_0000000000
   [junit4]   2> 385778 INFO  (zkCallback-2625-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/test_collection/state.json] for collection [test_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 385779 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37169/solr/test_collection_shard1_replica_n1/ shard1
   [junit4]   2> 385787 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr c:test_collection s:shard1 r:core_node2 x:test_collection_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 385789 INFO  (zkCallback-2625-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/test_collection/state.json] for collection [test_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 385794 INFO  (qtp233269725-8221) [n:127.0.0.1:37169_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=test&newCollection=true&name=test_collection_shard1_replica_n1&action=CREATE&numShards=2&collection=test_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1246
   [junit4]   2> 386263 INFO  (zkCallback-2625-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/test_collection/state.json] for collection [test_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 386264 INFO  (zkCallback-2603-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/test_collection/state.json] for collection [test_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 386266 INFO  (qtp1669237816-8214) [n:127.0.0.1:40507_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=test&newCollection=true&name=test_collection_shard1_replica_n3&action=CREATE&numShards=2&collection=test_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1718
   [junit4]   2> 442724 INFO  (qtp233269725-8220) [n:127.0.0.1:37169_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.test_collection.shard1.replica_n1:QUERY./select.requests&key=solr.core.test_collection.shard2.replica_n5:INDEX.sizeInBytes&key=solr.core.test_collection.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.test_collection.shard1.replica_n1:UPDATE./update.requests&key=solr.core.test_collection.shard2.replica_n5:QUERY./select.requests&key=solr.core.test_collection.shard2.replica_n5:UPDATE./update.requests} status=0 QTime=4
   [junit4]   2> 442732 INFO  (qtp233269725-8222) [n:127.0.0.1:37169_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.jvm:os.processCpuLoad&key=solr.node:CONTAINER.fs.coreRoot.usableSpace&key=solr.jvm:os.systemLoadAverage&key=solr.jvm:memory.heap.used} status=0 QTime=1
   [junit4]   2> 442742 INFO  (qtp1669237816-8217) [n:127.0.0.1:40507_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.test_collection.shard2.replica_n7:UPDATE./update.requests&key=solr.core.test_collection.shard2.replica_n7:INDEX.sizeInBytes&key=solr.core.test_collection.shard2.replica_n7:QUERY./select.requests&key=solr.core.test_collection.shard1.replica_n3:QUERY./select.requests&key=solr.core.test_collection.shard1.replica_n3:INDEX.sizeInBytes&key=solr.core.test_collection.shard1.replica_n3:UPDATE./update.requests} status=0 QTime=1
   [junit4]   2> 442743 INFO  (qtp1669237816-8213) [n:127.0.0.1:40507_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.jvm:os.processCpuLoad&key=solr.node:CONTAINER.fs.coreRoot.usableSpace&key=solr.jvm:os.systemLoadAverage&key=solr.jvm:memory.heap.used} status=0 QTime=0
   [junit4]   2> 474549 INFO  (TEST-SolrCloudReportersTest.testDefaultPlugins-seed#[94FC445D9EA98321]) [     ] o.a.s.SolrTestCaseJ4 ###Ending testDefaultPlugins
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=SolrCloudReportersTest -Dtests.method=testDefaultPlugins -Dtests.seed=94FC445D9EA98321 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=en-GB -Dtests.timezone=Europe/London -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   94.3s J1 | SolrCloudReportersTest.testDefaultPlugins <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: Timeout occurred while waiting response from server at: http://127.0.0.1:37169/solr
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([94FC445D9EA98321:DC7C168DC2E9A00C]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:692)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:266)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:369)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:297)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1171)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:934)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:866)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:214)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:231)
   [junit4]    > 	at org.apache.solr.metrics.reporters.solr.SolrCloudReportersTest.testDefaultPlugins(SolrCloudReportersTest.java:176)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:171)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:141)
   [junit4]    > 	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
   [junit4]    > 	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
   [junit4]    > 	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
   [junit4]    > 	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
   [junit4]    > 	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
   [junit4]    > 	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
   [junit4]    > 	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
   [junit4]    > 	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
   [junit4]    > 	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   [junit4]    > 	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:571)
   [junit4]    > 	... 53 more
   [junit4]   2> 474575 INFO  (TEST-SolrCloudReportersTest.testExplicitConfiguration-seed#[94FC445D9EA98321]) [     ] o.a.s.SolrTestCaseJ4 ###Starting testExplicitConfiguration
   [junit4]   2> 474690 INFO  (jetty-closer-2662-thread-2) [     ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1578317197
   [junit4]   2> 474690 INFO  (jetty-closer-2662-thread-2) [     ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40507_solr
   [junit4]   2> 474690 INFO  (jetty-closer-2662-thread-2) [     ] o.a.s.c.ZkController Publish this node as DOWN...
   [junit4]   2> 474691 INFO  (jetty-closer-2662-thread-2) [     ] o.a.s.c.ZkController Publish node=127.0.0.1:40507_solr as DOWN
   [junit4]   2> 474693 INFO  (coreCloseExecutor-2667-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.c.SolrCore [test_collection_shard1_replica_n3]  CLOSING SolrCore org.apache.solr.core.SolrCore@5369a94e
   [junit4]   2> 474693 INFO  (coreCloseExecutor-2667-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.test_collection.shard1.replica_n3 tag=SolrCore@5369a94e
   [junit4]   2> 474693 INFO  (coreCloseExecutor-2667-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2d7bb939: rootName = null, domain = solr.core.test_collection.shard1.replica_n3, service url = null, agent id = null] for registry solr.core.test_collection.shard1.replica_n3/com.codahale.metrics.MetricRegistry@61b5ae8
   [junit4]   2> 474693 INFO  (coreCloseExecutor-2667-thread-2) [n:127.0.0.1:40507_solr     ] o.a.s.c.SolrCore [test_collection_shard2_replica_n7]  CLOSING SolrCore org.apache.solr.core.SolrCore@4622bcb
   [junit4]   2> 474694 INFO  (zkCallback-2603-thread-4) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/test_collection/state.json] for collection [test_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 474694 INFO  (zkCallback-2625-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/test_collection/state.json] for collection [test_collection] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 474707 INFO  (coreCloseExecutor-2667-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.test_collection.shard1.leader tag=SolrCore@5369a94e
   [junit4]   2> 474707 INFO  (coreCloseExecutor-2667-thread-2) [n:127.0.0.1:40507_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.test_collection.shard2.replica_n7 tag=SolrCore@4622bcb
   [junit4]   2> 474707 INFO  (coreCloseExecutor-2667-thread-2) [n:127.0.0.1:40507_solr     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3ec6163: rootName = null, domain = solr.core.test_collection.shard2.replica_n7, service url = null, agent id = null] for registry solr.core.test_collection.shard2.replica_n7/com.codahale.metrics.MetricRegistry@4df88c87
   [junit4]   2> 474708 INFO  (coreCloseExecutor-2667-thread-1) [n:127.0.0.1:40507_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 474720 INFO  (coreCloseExecutor-2667-thread-2) [n:127.0.0.1:40507_solr     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.test_collection.shard2.leader tag=SolrCore@4622bcb
   [junit4]   2> 474721 INFO  (coreCloseExecutor-2667-thread-2) [n:127.0.0.1:40507_solr     ] o.a.s.u.DirectUpdateHandler2 Committing on IndexWriter.close()  ... SKIPPED (unnecessary).
   [junit4]   2> 474724 INFO  (jetty-closer-2662-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node tag=null
   [junit4]   2> 474724 INFO  (jetty-closer-2662-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7e5584b5: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node/com.codahale.metrics.MetricRegistry@33fb93c7
   [junit4]   2> 474730 INFO  (jetty-closer-2662-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm tag=null
   [junit4]   2> 474730 INFO  (jetty-closer-2662-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1b8c68f: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm/com.codahale.metrics.MetricRegistry@6e662df0
   [junit4]   2> 474733 INFO  (jetty-closer-2662-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty tag=null
   [junit4]   2> 474733 INFO  (jetty-closer-2662-thread-2) [     ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@48593c30: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty/com.codahale.metrics.MetricRegistry@211a0104
   [junit4]   2> 474733 INFO  (jetty-closer-2662-thread-2) [     ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster tag=null
   [junit4]   2> 474736 WARN  (jetty-launcher-2584-thread-1-SendThread(127.0.0.1:34879)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103aa706bb70008.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103aa706bb70008, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103aa706bb70008, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 474736 INFO  (zkCallback-2603-thread-4) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 474738 INFO  (zkCallback-2603-thread-5) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 474838 INFO  (jetty-closer-2662-thread-2) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@6db97e27{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:0}
   [junit4]   2> 474838 INFO  (jetty-closer-2662-thread-2) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@54295c37{/solr,null,STOPPED}
   [junit4]   2> 474838 INFO  (jetty-closer-2662-thread-2) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 477238 INFO  (zkCallback-2603-thread-5) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37169/solr/test_collection_shard2_replica_n5/
   [junit4]   2> 477239 ERROR (zkCallback-2603-thread-5) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.SyncStrategy No UpdateLog found - cannot sync
   [junit4]   2> 477239 INFO  (zkCallback-2603-thread-5) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 477239 INFO  (zkCallback-2603-thread-5) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/test_collection/leaders/shard2/leader after winning as /collections/test_collection/leader_elect/shard2/election/73089418788274182-core_node6-n_0000000001
   [junit4]   2> 477240 INFO  (zkCallback-2603-thread-5) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37169/solr/test_collection_shard2_replica_n5/ shard2
   [junit4]   2> 477240 INFO  (zkCallback-2603-thread-4) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/test_collection/state.json] for collection [test_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 478242 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr c:test_collection s:shard2 r:core_node6 x:test_collection_shard2_replica_n5 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 478243 INFO  (zkCallback-2603-thread-4) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeChildrenChanged path:/collections/test_collection/state.json] for collection [test_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 478245 INFO  (qtp233269725-8212) [n:127.0.0.1:37169_solr     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=test&newCollection=true&name=test_collection_shard2_replica_n5&action=CREATE&numShards=2&collection=test_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=93696
   [junit4]   2> 485203 ERROR (qtp233269725-8221) [n:127.0.0.1:37169_solr     ] o.a.s.s.SolrDispatchFilter Error processing the request. CoreContainer is either not initialized or shutting down.
   [junit4]   2> 485203 WARN  

[...truncated too long message...]

e-work
   [junit4]   2> 	2	/solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 	2	/solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 	2	/solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> 1970115 INFO  (SUITE-TestSubQueryTransformerDistrib-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:41189
   [junit4]   2> 1970115 INFO  (SUITE-TestSubQueryTransformerDistrib-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:41189
   [junit4]   2> 1970115 INFO  (SUITE-TestSubQueryTransformerDistrib-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 41189
   [junit4]   2> 1970115 INFO  (SUITE-TestSubQueryTransformerDistrib-seed#[94FC445D9EA98321]-worker) [     ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
   [junit4]   2> Mar 02, 2021 3:43:12 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=DeflateWithPresetCompressingStoredFieldsData(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=BEST_COMPRESSION, chunkSize=40, maxDocsPerChunk=6, blockShift=7), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=BEST_COMPRESSION, chunkSize=40, blockSize=7)), sim=Asserting(RandomSimilarity(queryNorm=true): {}), locale=mk-MK, timezone=Indian/Maldives
   [junit4]   2> NOTE: Linux 5.4.0-45-generic amd64/Oracle Corporation 1.8.0_281 (64-bit)/cpus=4,threads=1,free=139859048,total=513802240
   [junit4]   2> NOTE: All tests run in this JVM: [TestCollectionAPI, SuggesterFSTTest, DistribDocExpirationUpdateProcessorTest, TestUpdate, DistributedIntervalFacetingTest, TestMacros, AuditLoggerPluginTest, AsyncCallRequestStatusResponseTest, TestCoreContainer, DocValuesNotIndexedTest, CollectionsAPIAsyncDistributedZkTest, ChangedSchemaMergeTest, CoreAdminHandlerTest, TestSweetSpotSimilarityFactory, DateRangeFieldTest, BlockJoinFacetRandomTest, PeerSyncTest, TestExactSharedStatsCache, TestCrossCoreJoin, SpellPossibilityIteratorTest, SolrCloudReportersTest, JavaBinAtomicUpdateMultivalueTest, TestSafeXMLParsing, BlockCacheTest, PurgeGraphTest, ZkStateWriterTest, ByteBuffersDirectoryFactoryTest, TestFacetMethods, TestBulkSchemaAPI, TestAddFieldRealTimeGet, RegexBoostProcessorTest, JWTAuthPluginIntegrationTest, TestCollationFieldDocValues, ShardRoutingTest, PluginInfoTest, DistributedTermsComponentTest, TestSchemaManager, TestCloudJSONFacetSKGEquiv, TermsComponentTest, TestEmbeddedSolrServerAdminHandler, URLClassifyProcessorTest, TestJavabinTupleStreamParser, TestDFRSimilarityFactory, TestPullReplicaErrorHandling, CustomCollectionTest, DistributedFacetExistsSmallTest, OverseerCollectionConfigSetProcessorTest, TestReversedWildcardFilterFactory, TestInPlaceUpdatesRequiredField, TriggerEventQueueTest, CreateRoutedAliasTest, TestSolr4Spatial2, TestRangeQuery, TestMiniSolrCloudClusterSSL, HdfsThreadLeakTest, DateMathParserTest, WrapperMergePolicyFactoryTest, TestFieldCollectionResource, SolrGraphiteReporterTest, TestLFUCache, TestStreamBody, TestDistribPackageStore, StressHdfsTest, TestSerializedLuceneMatchVersion, TestSubQueryTransformerDistrib]
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSubQueryTransformerDistrib -Dtests.seed=94FC445D9EA98321 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=mk-MK -Dtests.timezone=Indian/Maldives -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   0.00s J1 | TestSubQueryTransformerDistrib (suite) <<<
   [junit4]    > Throwable #1: org.apache.solr.client.solrj.SolrServerException: Timeout occurred while waiting response from server at: http://127.0.0.1:33487/solr
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([94FC445D9EA98321]:0)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:692)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:266)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBSolrClient.doRequest(LBSolrClient.java:369)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.LBSolrClient.request(LBSolrClient.java:297)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1171)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:934)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:866)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:214)
   [junit4]    > 	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:231)
   [junit4]    > 	at org.apache.solr.response.transform.TestSubQueryTransformerDistrib.setupCluster(TestSubQueryTransformerDistrib.java:83)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]    > Caused by: java.net.SocketTimeoutException: Read timed out
   [junit4]    > 	at java.net.SocketInputStream.socketRead0(Native Method)
   [junit4]    > 	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:171)
   [junit4]    > 	at java.net.SocketInputStream.read(SocketInputStream.java:141)
   [junit4]    > 	at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
   [junit4]    > 	at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
   [junit4]    > 	at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
   [junit4]    > 	at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
   [junit4]    > 	at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
   [junit4]    > 	at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
   [junit4]    > 	at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
   [junit4]    > 	at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
   [junit4]    > 	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
   [junit4]    > 	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
   [junit4]    > 	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
   [junit4]    > 	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
   [junit4]    > 	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
   [junit4]    > 	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
   [junit4]    > 	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:571)
   [junit4]    > 	... 36 more
   [junit4] Completed [439/934 (2!)] on J1 in 96.01s, 0 tests, 1 error <<< FAILURES!

[...truncated 52481 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:652: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:507: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:494: Source checkout is dirty (unversioned/missing files) after running tests!!! Offending files:
* solr/contrib/analytics/src/test-files/solr/filestore
* solr/contrib/analytics/src/test-files/solr/userfiles
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/collection1/conf/dataimport.properties
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/filestore
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/userfiles
* solr/contrib/dataimporthandler/src/test-files/dih/solr/filestore
* solr/contrib/dataimporthandler/src/test-files/dih/solr/userfiles
* solr/contrib/extraction/src/test-files/extraction/solr/filestore
* solr/contrib/extraction/src/test-files/extraction/solr/userfiles
* solr/contrib/langid/src/test-files/langid/solr/filestore
* solr/contrib/langid/src/test-files/langid/solr/userfiles
* solr/contrib/ltr/src/test-files/solr/filestore
* solr/contrib/ltr/src/test-files/solr/userfiles
* solr/contrib/prometheus-exporter/src/test-files/solr/filestore
* solr/contrib/prometheus-exporter/src/test-files/solr/userfiles
* solr/contrib/velocity/src/test-files/velocity/solr/filestore
* solr/contrib/velocity/src/test-files/velocity/solr/userfiles
* solr/core/src/test-files/solr/filestore
* solr/core/src/test-files/solr/userfiles
* solr/solrj/src/test-files/solrj/solr/multicore/filestore
* solr/solrj/src/test-files/solrj/solr/multicore/userfiles

Total time: 297 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any

[JENKINS] Lucene » Lucene-Solr-NightlyTests-8.x - Build # 197 - Still Failing!

Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://ci-builds.apache.org/job/Lucene/job/Lucene-Solr-NightlyTests-8.x/197/

1 tests failed.
FAILED:  org.apache.solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest.test

Error Message:
shard1 is not consistent.  Got 1447 from http://127.0.0.1:42471/iua/j/collection1_shard1_replica_n3 (previous client) and got 1449 from http://127.0.0.1:46689/iua/j/collection1_shard1_replica_n1

Stack Trace:
java.lang.AssertionError: shard1 is not consistent.  Got 1447 from http://127.0.0.1:42471/iua/j/collection1_shard1_replica_n3 (previous client) and got 1449 from http://127.0.0.1:46689/iua/j/collection1_shard1_replica_n1
	at __randomizedtesting.SeedInfo.seed([24FE8A9D4E8F9FD9:ACAAB547E073F221]:0)
	at org.junit.Assert.fail(Assert.java:89)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1486)
	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1465)
	at org.apache.solr.cloud.TlogReplayBufferedWhileIndexingTest.test(TlogReplayBufferedWhileIndexingTest.java:109)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1090)
	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1061)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 13669 lines...]
   [junit4] Suite: org.apache.solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest
   [junit4]   2> 332027 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.a.s.SolrTestCase Setting 'solr.default.confdir' system property to test-framework derived value of '/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/server/solr/configsets/_default/conf'
   [junit4]   2> 332027 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/data-dir-48-001
   [junit4]   2> 332027 WARN  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
   [junit4]   2> 332028 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 332030 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
   [junit4]   2> 332030 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 332030 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /iua/j
   [junit4]   1> Formatting using clusterid: testClusterID
   [junit4]   2> 334297 WARN  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 334324 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 334325 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 334325 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 334325 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 334327 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@76500e6e{static,/static,jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,AVAILABLE}
   [junit4]   2> 334515 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@727e6477{hdfs,/,file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/jetty-localhost_localdomain-38199-hadoop-hdfs-3_2_0-tests_jar-_-any-5000705656060785588/webapp/,AVAILABLE}{jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/hdfs}
   [junit4]   2> 334515 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.AbstractConnector Started ServerConnector@11d4cdc4{HTTP/1.1, (http/1.1)}{localhost.localdomain:38199}
   [junit4]   2> 334515 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.Server Started @334596ms
   [junit4]   2> 335452 WARN  (StorageLocationChecker thread 0) [     ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
   [junit4]   2> 335513 WARN  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
   [junit4]   2> 335518 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 335521 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 335521 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 335521 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 335524 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33cbda32{static,/static,jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,AVAILABLE}
   [junit4]   2> 335627 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.h.ContextHandler Started o.e.j.w.WebAppContext@4572d17{datanode,/,file:///home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/jetty-localhost-36989-hadoop-hdfs-3_2_0-tests_jar-_-any-5505858018378618619/webapp/,AVAILABLE}{jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/datanode}
   [junit4]   2> 335627 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.AbstractConnector Started ServerConnector@3eee4207{HTTP/1.1, (http/1.1)}{localhost:36989}
   [junit4]   2> 335627 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.Server Started @335707ms
   [junit4]   2> 338028 INFO  (Block report processor) [     ] BlockStateChange BLOCK* processReport 0x10a73049ebb3a770: Processing first storage report for DS-db8c1ab0-193c-4c16-a79d-1aed26d51a94 from datanode 64c12b4a-6e47-4a42-98e9-db4df862592f
   [junit4]   2> 338031 INFO  (Block report processor) [     ] BlockStateChange BLOCK* processReport 0x10a73049ebb3a770: from storage DS-db8c1ab0-193c-4c16-a79d-1aed26d51a94 node DatanodeRegistration(127.0.0.1:41463, datanodeUuid=64c12b4a-6e47-4a42-98e9-db4df862592f, infoPort=32889, infoSecurePort=0, ipcPort=42689, storageInfo=lv=-57;cid=testClusterID;nsid=2025423177;c=1614607809384), blocks: 0, hasStaleStorage: true, processing time: 3 msecs, invalidatedBlocks: 0
   [junit4]   2> 338031 INFO  (Block report processor) [     ] BlockStateChange BLOCK* processReport 0x10a73049ebb3a770: Processing first storage report for DS-a1463ac9-8c53-465d-a526-47031f16dbef from datanode 64c12b4a-6e47-4a42-98e9-db4df862592f
   [junit4]   2> 338031 INFO  (Block report processor) [     ] BlockStateChange BLOCK* processReport 0x10a73049ebb3a770: from storage DS-a1463ac9-8c53-465d-a526-47031f16dbef node DatanodeRegistration(127.0.0.1:41463, datanodeUuid=64c12b4a-6e47-4a42-98e9-db4df862592f, infoPort=32889, infoSecurePort=0, ipcPort=42689, storageInfo=lv=-57;cid=testClusterID;nsid=2025423177;c=1614607809384), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0
   [junit4]   2> 338149 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 338157 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer client port: 0.0.0.0/0.0.0.0:0
   [junit4]   2> 338157 INFO  (ZkTestServer Run Thread) [     ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 338158 WARN  (ZkTestServer Run Thread) [     ] o.a.z.s.ServerCnxnFactory maxCnxns is not configured, using default value 0.
   [junit4]   2> 338257 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer start zk server on port: 38391
   [junit4]   2> 338257 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:38391
   [junit4]   2> 338257 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:38391
   [junit4]   2> 338257 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 38391
   [junit4]   2> 338259 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 338267 INFO  (zkConnectionManagerCallback-3421-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 338268 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 338273 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 338279 INFO  (zkConnectionManagerCallback-3423-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 338279 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 338286 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 338288 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
   [junit4]   2> 338290 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 338293 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 338294 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 338297 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 338298 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 338299 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 338300 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 338301 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 338302 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer put /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 338303 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
   [junit4]   2> 338398 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 338398 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 338398 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 338400 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 338400 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 338400 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 338400 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@668c2eea{/iua/j,null,AVAILABLE}
   [junit4]   2> 338401 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.e.j.s.AbstractConnector Started ServerConnector@2d0728ec{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:37313}
   [junit4]   2> 338401 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.e.j.s.Server Started @338481ms
   [junit4]   2> 338401 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost.localdomain:44835/hdfs__localhost.localdomain_44835__home_jenkins_jenkins-slave_workspace_Lucene_Lucene-Solr-NightlyTests-8.x_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001_tempDir-002_control_data, replicaType=NRT, hostContext=/iua/j, hostPort=37313, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/control-001/cores}
   [junit4]   2> 338401 ERROR (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 338401 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 338401 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 338401 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 338401 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 338401 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-01T14:10:14.224Z
   [junit4]   2> 338402 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 338409 INFO  (zkConnectionManagerCallback-3425-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 338409 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 338511 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 338511 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/control-001/solr.xml
   [junit4]   2> 338514 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 338514 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 338515 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 340612 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 340613 WARN  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@c608caf[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 340613 WARN  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@c608caf[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 340725 WARN  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@62d5710f[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 340725 WARN  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@62d5710f[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 340727 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38391/solr
   [junit4]   2> 340730 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 340732 INFO  (zkConnectionManagerCallback-3436-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 340732 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 340733 WARN  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]-SendThread(127.0.0.1:38391)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103a50affaa0003.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103a50affaa0003, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103a50affaa0003, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 340835 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 340836 INFO  (zkConnectionManagerCallback-3438-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 340837 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 341040 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37313_iua%2Fj
   [junit4]   2> 341040 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.Overseer Overseer (id=73083485625647108-127.0.0.1:37313_iua%2Fj-n_0000000000) starting
   [junit4]   2> 341047 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37313_iua%2Fj
   [junit4]   2> 341048 INFO  (OverseerStateUpdate-73083485625647108-127.0.0.1:37313_iua%2Fj-n_0000000000) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:37313_iua%2Fj
   [junit4]   2> 341050 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 341050 WARN  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 341052 INFO  (OverseerStateUpdate-73083485625647108-127.0.0.1:37313_iua%2Fj-n_0000000000) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 341069 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 341090 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d
   [junit4]   2> 341099 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d
   [junit4]   2> 341099 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d
   [junit4]   2> 341101 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/control-001/cores
   [junit4]   2> 341119 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 341126 INFO  (zkConnectionManagerCallback-3455-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 341126 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 341128 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 341131 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38391/solr ready
   [junit4]   2> 341137 INFO  (OverseerThreadFactory-3445-thread-1-processing-n:127.0.0.1:37313_iua%2Fj) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
   [junit4]   2> 341260 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj    x:control_collection_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 341260 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj    x:control_collection_shard1_replica_n1 ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 342276 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 342289 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 342293 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 342334 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from configset conf1, trusted=true
   [junit4]   2> 342334 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d
   [junit4]   2> 342343 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost.localdomain:44835/solr_hdfs_home
   [junit4]   2> 342343 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 342343 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[hdfs://localhost.localdomain:44835/solr_hdfs_home/control_collection/core_node2/data/]
   [junit4]   2> 342346 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:44835/solr_hdfs_home/control_collection/core_node2/data/snapshot_metadata
   [junit4]   2> 342362 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 342362 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
   [junit4]   2> 342362 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 342608 WARN  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.s.h.HdfsDirectory HDFS support in Solr has been deprecated as of 8.6. See SOLR-14021 for details.
   [junit4]   2> 342608 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 342615 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:44835/solr_hdfs_home/control_collection/core_node2/data
   [junit4]   2> 342641 WARN  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.s.h.HdfsDirectory HDFS support in Solr has been deprecated as of 8.6. See SOLR-14021 for details.
   [junit4]   2> 342656 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:44835/solr_hdfs_home/control_collection/core_node2/data/index
   [junit4]   2> 342667 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 342667 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
   [junit4]   2> 342667 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 342674 WARN  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.s.h.HdfsDirectory HDFS support in Solr has been deprecated as of 8.6. See SOLR-14021 for details.
   [junit4]   2> 342674 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 342674 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=0.748046875, floorSegmentMB=0.3388671875, forceMergeDeletesPctAllowed=17.112255088754587, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0, deletesPctAllowed=24.529942946254476
   [junit4]   2> 343630 WARN  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 343827 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 343827 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 343828 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 343842 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 10000ms; 
   [junit4]   2> 343842 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: if uncommitted for 3000ms; 
   [junit4]   2> 343847 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@69630194
   [junit4]   2> 343965 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 343966 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 343968 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 343969 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693039009246216192
   [junit4]   2> 343980 INFO  (searcherExecutor-3457-thread-1-processing-n:127.0.0.1:37313_iua%2Fj x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SolrCore [control_collection_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 343995 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 343995 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/control_collection/leaders/shard1
   [junit4]   2> 344003 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 344003 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 344003 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37313/iua/j/control_collection_shard1_replica_n1/
   [junit4]   2> 344004 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 344005 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.SyncStrategy http://127.0.0.1:37313/iua/j/control_collection_shard1_replica_n1/ has no replicas
   [junit4]   2> 344005 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/control_collection/leaders/shard1/leader after winning as /collections/control_collection/leader_elect/shard1/election/73083485625647108-core_node2-n_0000000000
   [junit4]   2> 344008 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37313/iua/j/control_collection_shard1_replica_n1/ shard1
   [junit4]   2> 344011 INFO  (zkCallback-3437-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 344013 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj c:control_collection s:shard1  x:control_collection_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 344016 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2755
   [junit4]   2> 344019 INFO  (qtp218688755-6831) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 344120 INFO  (zkCallback-3437-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 344120 INFO  (zkCallback-3437-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 344121 INFO  (qtp218688755-6831) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:37313_iua%252Fj&wt=javabin&version=2} status=0 QTime=2989
   [junit4]   2> 344122 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Waiting to see 1 active replicas in collection: control_collection
   [junit4]   2> 344230 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 344232 INFO  (zkConnectionManagerCallback-3466-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 344232 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 344234 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 344235 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38391/solr ready
   [junit4]   2> 344235 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 344240 INFO  (OverseerThreadFactory-3445-thread-2-processing-n:127.0.0.1:37313_iua%2Fj) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
   [junit4]   2> 344240 INFO  (OverseerCollectionConfigSetProcessor-73083485625647108-127.0.0.1:37313_iua%2Fj-n_0000000000) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 344445 WARN  (OverseerThreadFactory-3445-thread-2-processing-n:127.0.0.1:37313_iua%2Fj) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
   [junit4]   2> 344447 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
   [junit4]   2> 344450 INFO  (qtp218688755-6833) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=214
   [junit4]   2> 344451 INFO  (watches-3463-thread-1) [     ] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1
   [junit4]   2> 344452 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1
   [junit4]   2> 344452 INFO  (watches-3463-thread-1) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 344452 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 344452 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1
   [junit4]   2> 344452 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 344452 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.SolrCloudTestCase active slice count: 1 expected: 1
   [junit4]   2> 344452 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.SolrCloudTestCase active replica count: 0 expected replica count: 0
   [junit4]   2> 344452 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.AbstractFullDistribZkTestBase Creating jetty instances pullReplicaCount=0 numOtherReplicas=2
   [junit4]   2> 344593 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/shard-1-001 of type NRT for shard1
   [junit4]   2> 344598 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 344598 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 344598 INFO  (closeThreadPool-3467-thread-1) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 344705 INFO  (closeThreadPool-3467-thread-1) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 344705 INFO  (closeThreadPool-3467-thread-1) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 344705 INFO  (closeThreadPool-3467-thread-1) [     ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 344714 INFO  (closeThreadPool-3467-thread-1) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b20848b{/iua/j,null,AVAILABLE}
   [junit4]   2> 344714 INFO  (closeThreadPool-3467-thread-1) [     ] o.e.j.s.AbstractConnector Started ServerConnector@66a895d4{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:42471}
   [junit4]   2> 344714 INFO  (closeThreadPool-3467-thread-1) [     ] o.e.j.s.Server Started @344794ms
   [junit4]   2> 344714 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost.localdomain:44835/hdfs__localhost.localdomain_44835__home_jenkins_jenkins-slave_workspace_Lucene_Lucene-Solr-NightlyTests-8.x_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001_tempDir-002_jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/iua/j, hostPort=42471, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/shard-1-001/cores}
   [junit4]   2> 344714 ERROR (closeThreadPool-3467-thread-1) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 344714 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 344714 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 344714 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 344714 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 344714 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-01T14:10:20.537Z
   [junit4]   2> 344717 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 344729 INFO  (zkConnectionManagerCallback-3469-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 344729 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 344801 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/shard-2-001 of type NRT for shard1
   [junit4]   2> 344802 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
   [junit4]   2> 344802 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 2 ...
   [junit4]   2> 344802 INFO  (closeThreadPool-3467-thread-2) [     ] o.e.j.s.Server jetty-9.4.34.v20201102; built: 2020-11-02T14:15:39.302Z; git: e46af88704a893fc12cb0e3bf46e2c7b48a009e7; jvm 1.8.0_281-b09
   [junit4]   2> 344841 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 344841 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/shard-1-001/solr.xml
   [junit4]   2> 344844 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 344844 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 345010 INFO  (closeThreadPool-3467-thread-2) [     ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 345010 INFO  (closeThreadPool-3467-thread-2) [     ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 345011 INFO  (closeThreadPool-3467-thread-2) [     ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 345011 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 345011 INFO  (closeThreadPool-3467-thread-2) [     ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@33a3c24d{/iua/j,null,AVAILABLE}
   [junit4]   2> 345013 INFO  (closeThreadPool-3467-thread-2) [     ] o.e.j.s.AbstractConnector Started ServerConnector@61e67609{HTTP/1.1, (http/1.1, h2c)}{127.0.0.1:46689}
   [junit4]   2> 345013 INFO  (closeThreadPool-3467-thread-2) [     ] o.e.j.s.Server Started @345093ms
   [junit4]   2> 345013 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=hdfs://localhost.localdomain:44835/hdfs__localhost.localdomain_44835__home_jenkins_jenkins-slave_workspace_Lucene_Lucene-Solr-NightlyTests-8.x_checkout_solr_build_solr-core_test_J2_temp_solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001_tempDir-002_jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/iua/j, hostPort=46689, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/shard-2-001/cores}
   [junit4]   2> 345013 ERROR (closeThreadPool-3467-thread-2) [     ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 345013 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 345013 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.9.0
   [junit4]   2> 345013 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 345013 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr
   [junit4]   2> 345013 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2021-03-01T14:10:20.836Z
   [junit4]   2> 345014 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 345015 INFO  (zkConnectionManagerCallback-3474-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 345015 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 345017 WARN  (closeThreadPool-3467-thread-2-SendThread(127.0.0.1:38391)) [     ] o.a.z.ClientCnxn An exception was thrown while closing send thread for session 0x103a50affaa0008.
   [junit4]   2>           => EndOfStreamException: Unable to read additional data from server sessionid 0x103a50affaa0008, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
   [junit4]   2> org.apache.zookeeper.ClientCnxn$EndOfStreamException: Unable to read additional data from server sessionid 0x103a50affaa0008, likely server has closed socket
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350) ~[zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1275) [zookeeper-3.6.2.jar:3.6.2]
   [junit4]   2> 345119 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
   [junit4]   2> 345119 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/shard-2-001/solr.xml
   [junit4]   2> 345121 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 345121 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 345123 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 345276 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 345277 WARN  (closeThreadPool-3467-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@5e15a363[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 345277 WARN  (closeThreadPool-3467-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@5e15a363[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 345280 WARN  (closeThreadPool-3467-thread-1) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@24af7fee[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 345281 WARN  (closeThreadPool-3467-thread-1) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@24af7fee[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 345282 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38391/solr
   [junit4]   2> 345282 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 345285 INFO  (zkConnectionManagerCallback-3485-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 345285 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 345630 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 345631 INFO  (zkConnectionManagerCallback-3487-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 345631 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 345633 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 345638 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.c.ZkController Publish node=127.0.0.1:42471_iua%2Fj as DOWN
   [junit4]   2> 345639 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 345639 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42471_iua%2Fj
   [junit4]   2> 345640 INFO  (zkCallback-3437-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 345642 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 345642 WARN  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 345651 INFO  (zkCallback-3486-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 345654 INFO  (zkCallback-3465-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 345658 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 345676 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d
   [junit4]   2> 345686 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d
   [junit4]   2> 345686 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d
   [junit4]   2> 345687 INFO  (closeThreadPool-3467-thread-1) [n:127.0.0.1:42471_iua%2Fj     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/shard-1-001/cores
   [junit4]   2> 345703 INFO  (closeThreadPool-3467-thread-1) [     ] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:42471_iua%2Fj
   [junit4]   2> 346243 INFO  (OverseerCollectionConfigSetProcessor-73083485625647108-127.0.0.1:37313_iua%2Fj-n_0000000000) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 346567 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=false]
   [junit4]   2> 346569 WARN  (closeThreadPool-3467-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@79f82aa2[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 346570 WARN  (closeThreadPool-3467-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@79f82aa2[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 346573 WARN  (closeThreadPool-3467-thread-2) [     ] o.e.j.u.s.S.config Trusting all certificates configured for Client@2749a447[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 346573 WARN  (closeThreadPool-3467-thread-2) [     ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@2749a447[provider=null,keyStore=null,trustStore=null]
   [junit4]   2> 346574 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38391/solr
   [junit4]   2> 346575 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 346579 INFO  (zkConnectionManagerCallback-3503-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 346579 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 346782 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
   [junit4]   2> 346783 INFO  (zkConnectionManagerCallback-3505-thread-1) [     ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 346783 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
   [junit4]   2> 346788 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 346792 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.c.ZkController Publish node=127.0.0.1:46689_iua%2Fj as DOWN
   [junit4]   2> 346792 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 346793 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46689_iua%2Fj
   [junit4]   2> 346794 INFO  (zkCallback-3465-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 346795 INFO  (zkCallback-3437-thread-2) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 346796 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.p.PackageLoader /packages.json updated to version -1
   [junit4]   2> 346796 WARN  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.c.CoreContainer Not all security plugins configured!  authentication=disabled authorization=disabled.  Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external.  See https://s.apache.org/solrsecurity for more info
   [junit4]   2> 346804 INFO  (zkCallback-3486-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 346804 INFO  (zkCallback-3504-thread-1) [     ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
   [junit4]   2> 346809 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 346827 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d
   [junit4]   2> 346837 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d
   [junit4]   2> 346837 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d
   [junit4]   2> 346838 INFO  (closeThreadPool-3467-thread-2) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/shard-2-001/cores
   [junit4]   2> 346874 INFO  (closeThreadPool-3467-thread-2) [     ] o.a.s.c.AbstractFullDistribZkTestBase waitForLiveNode: 127.0.0.1:46689_iua%2Fj
   [junit4]   2> 346895 INFO  (OverseerThreadFactory-3445-thread-3-processing-n:127.0.0.1:37313_iua%2Fj) [n:127.0.0.1:37313_iua%2Fj c:collection1 s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:46689_iua%2Fj for creating new replica of shard shard1 for collection collection1
   [junit4]   2> 346897 INFO  (OverseerThreadFactory-3445-thread-3-processing-n:127.0.0.1:37313_iua%2Fj) [n:127.0.0.1:37313_iua%2Fj c:collection1 s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 346910 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj    x:collection1_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 347930 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 347943 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 347947 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 347962 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from configset conf1, trusted=true
   [junit4]   2> 347962 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@54b04e8d
   [junit4]   2> 347962 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost.localdomain:44835/solr_hdfs_home
   [junit4]   2> 347962 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
   [junit4]   2> 347962 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001/shard-2-001/cores/collection1_shard1_replica_n1], dataDir=[hdfs://localhost.localdomain:44835/solr_hdfs_home/collection1/core_node2/data/]
   [junit4]   2> 347963 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:44835/solr_hdfs_home/collection1/core_node2/data/snapshot_metadata
   [junit4]   2> 347971 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 347971 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
   [junit4]   2> 347971 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 347978 WARN  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.s.h.HdfsDirectory HDFS support in Solr has been deprecated as of 8.6. See SOLR-14021 for details.
   [junit4]   2> 347978 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 347979 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:44835/solr_hdfs_home/collection1/core_node2/data
   [junit4]   2> 347991 WARN  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.s.h.HdfsDirectory HDFS support in Solr has been deprecated as of 8.6. See SOLR-14021 for details.
   [junit4]   2> 348002 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost.localdomain:44835/solr_hdfs_home/collection1/core_node2/data/index
   [junit4]   2> 348009 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Number of slabs of block cache [1] with direct memory allocation set to [true]
   [junit4]   2> 348009 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Block cache target memory usage, slab size of [4194304] will allocate [1] slabs and use ~[4194304] bytes
   [junit4]   2> 348009 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.HdfsDirectoryFactory Creating new single instance HDFS BlockCache
   [junit4]   2> 348016 WARN  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.s.h.HdfsDirectory HDFS support in Solr has been deprecated as of 8.6. See SOLR-14021 for details.
   [junit4]   2> 348016 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.s.b.BlockDirectory Block cache on write is disabled
   [junit4]   2> 348016 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=21, maxMergedSegmentMB=0.748046875, floorSegmentMB=0.3388671875, forceMergeDeletesPctAllowed=17.112255088754587, segmentsPerTier=40.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0, deletesPctAllowed=24.529942946254476
   [junit4]   2> 348070 WARN  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
   [junit4]   2> 348130 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
   [junit4]   2> 348130 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 348130 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=2
   [junit4]   2> 348142 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: if uncommitted for 10000ms; 
   [junit4]   2> 348142 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: if uncommitted for 3000ms; 
   [junit4]   2> 348144 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@5e5efcf
   [junit4]   2> 348162 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 348162 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 348162 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000 ms
   [junit4]   2> 348163 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1693039013643943936
   [junit4]   2> 348169 INFO  (searcherExecutor-3516-thread-1-processing-n:127.0.0.1:46689_iua%2Fj x:collection1_shard1_replica_n1 c:collection1 s:shard1) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [collection1_shard1_replica_n1]  Registered new searcher autowarm time: 0 ms
   [junit4]   2> 348358 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node2=0}, version=0}
   [junit4]   2> 348358 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
   [junit4]   2> 348362 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 348362 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 348362 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46689/iua/j/collection1_shard1_replica_n1/
   [junit4]   2> 348363 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 348363 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy http://127.0.0.1:46689/iua/j/collection1_shard1_replica_n1/ has no replicas
   [junit4]   2> 348363 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/73083485625647116-core_node2-n_0000000000
   [junit4]   2> 348366 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46689/iua/j/collection1_shard1_replica_n1/ shard1
   [junit4]   2> 348468 INFO  (zkCallback-3504-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 348468 INFO  (zkCallback-3504-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 348470 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 348474 INFO  (qtp1708781162-6913) [n:127.0.0.1:46689_iua%2Fj     ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n1&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1563
   [junit4]   2> 348480 INFO  (qtp218688755-6831) [n:127.0.0.1:37313_iua%2Fj c:collection1    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:46689_iua%252Fj&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1598
   [junit4]   2> 348573 INFO  (zkCallback-3504-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 348573 INFO  (zkCallback-3504-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 348574 INFO  (zkCallback-3504-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 348889 INFO  (OverseerCollectionConfigSetProcessor-73083485625647108-127.0.0.1:37313_iua%2Fj-n_0000000000) [n:127.0.0.1:37313_iua%2Fj     ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
   [junit4]   2> 348890 INFO  (OverseerThreadFactory-3445-thread-4-processing-n:127.0.0.1:37313_iua%2Fj) [n:127.0.0.1:37313_iua%2Fj c:collection1 s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:42471_iua%2Fj for creating new replica of shard shard1 for collection collection1
   [junit4]   2> 348892 INFO  (OverseerThreadFactory-3445-thread-4-processing-n:127.0.0.1:37313_iua%2Fj) [n:127.0.0.1:37313_iua%2Fj c:collection1 s:shard1   ] o.a.s.c.a.c.AddReplicaCmd Returning CreateReplica command.
   [junit4]   2> 348901 INFO  (qtp1654070622-6898) [n:127.0.0.1:42471_iua%2Fj    x:collection1_shard1_replica_n3 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n3&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 349007 INFO  (zkCallback-3504-thread-3) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 349007 INFO  (zkCallback-3504-thread-1) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 349007 INFO  (zkCallback-3504-thread-2) [     ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
   [junit4]   2> 349914 INFO  (qtp1654070622-6898) [n:127.0.0.1:42471_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n3 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.9.0
   [junit4]   2> 349989 INFO  (qtp1654070622-6898) [n:127.0.0.1:42471_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n3 ] o.a.s.s.IndexSchema Schema name=test
   [junit4]   2> 349993 INFO  (qtp1654070622-6898) [n:127.0.0.1:42471_iua%2Fj c:collection1 s:shard1  x:collection1_shard1_replica_n3 ] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
   [junit4]   2> 350024 INFO  (qtp1654070622-6898) [n:127.0.0.1:42471_iua%2Fj c:collection1 s:shard1  x:collection1

[...truncated too long message...]

ops.json
   [junit4]   2> 	3	/solr/overseer_elect/election/73083485625647108-127.0.0.1:37313_iua%2Fj-n_0000000000
   [junit4]   2> 	2	/solr/collections/collection1/leader_elect/shard1/election/73083485625647116-core_node2-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	77	/solr/collections/collection1/state.json
   [junit4]   2> 	41	/solr/overseer/queue
   [junit4]   2> 	24	/solr/live_nodes
   [junit4]   2> 	14	/solr/overseer/collection-queue-work
   [junit4]   2> 	6	/solr/collections
   [junit4]   2> 	4	/solr/collections/control_collection/state.json
   [junit4]   2> 
   [junit4]   2> 858560 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:38391
   [junit4]   2> 858560 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:38391
   [junit4]   2> 858560 INFO  (TEST-HdfsTlogReplayBufferedWhileIndexingTest.test-seed#[24FE8A9D4E8F9FD9]) [     ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 38391
   [junit4]   2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=HdfsTlogReplayBufferedWhileIndexingTest -Dtests.method=test -Dtests.seed=24FE8A9D4E8F9FD9 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/test-data/enwiki.random.lines.txt -Dtests.locale=is-IS -Dtests.timezone=Pacific/Guam -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE  520s J2 | HdfsTlogReplayBufferedWhileIndexingTest.test <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: shard1 is not consistent.  Got 1447 from http://127.0.0.1:42471/iua/j/collection1_shard1_replica_n3 (previous client) and got 1449 from http://127.0.0.1:46689/iua/j/collection1_shard1_replica_n1
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([24FE8A9D4E8F9FD9:ACAAB547E073F221]:0)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1486)
   [junit4]    > 	at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1465)
   [junit4]    > 	at org.apache.solr.cloud.TlogReplayBufferedWhileIndexingTest.test(TlogReplayBufferedWhileIndexingTest.java:109)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1090)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:1061)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> 858567 WARN  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
   [junit4]   2> 858573 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.w.WebAppContext@4572d17{datanode,/,null,STOPPED}{jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/datanode}
   [junit4]   2> 858573 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@3eee4207{HTTP/1.1, (http/1.1)}{localhost:0}
   [junit4]   2> 858574 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 858574 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@33cbda32{static,/static,jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,STOPPED}
   [junit4]   2> 858575 WARN  (BP-444831082-127.0.0.1-1614607809384 heartbeating to localhost.localdomain/127.0.0.1:44835) [     ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
   [junit4]   2> 858575 WARN  (BP-444831082-127.0.0.1-1614607809384 heartbeating to localhost.localdomain/127.0.0.1:44835) [     ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-444831082-127.0.0.1-1614607809384 (Datanode Uuid 64c12b4a-6e47-4a42-98e9-db4df862592f) service to localhost.localdomain/127.0.0.1:44835
   [junit4]   2> 858604 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.w.WebAppContext@727e6477{hdfs,/,null,STOPPED}{jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/hdfs}
   [junit4]   2> 858604 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.AbstractConnector Stopped ServerConnector@11d4cdc4{HTTP/1.1, (http/1.1)}{localhost.localdomain:0}
   [junit4]   2> 858604 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 858604 INFO  (SUITE-HdfsTlogReplayBufferedWhileIndexingTest-seed#[24FE8A9D4E8F9FD9]-worker) [     ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@76500e6e{static,/static,jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-3.2.0-tests.jar!/webapps/static,STOPPED}
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/solr/build/solr-core/test/J2/temp/solr.cloud.hdfs.HdfsTlogReplayBufferedWhileIndexingTest_24FE8A9D4E8F9FD9-001
   [junit4]   2> Mar 01, 2021 2:18:54 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 16 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene87): {rnd_b=PostingsFormat(name=LuceneFixedGap), a_t=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene84)), _root_=PostingsFormat(name=Asserting), a_i=PostingsFormat(name=LuceneFixedGap), id=PostingsFormat(name=LuceneFixedGap)}, docValues:{_version_=DocValuesFormat(name=Lucene80)}, maxPointsInLeafNode=968, maxMBSortInHeap=6.584207837879138, sim=Asserting(RandomSimilarity(queryNorm=false): {}), locale=is-IS, timezone=Pacific/Guam
   [junit4]   2> NOTE: Linux 4.15.0-112-generic amd64/Oracle Corporation 1.8.0_281 (64-bit)/cpus=4,threads=6,free=333655544,total=524288000
   [junit4]   2> NOTE: All tests run in this JVM: [RemoteQueryErrorTest, TestWordDelimiterFilterFactory, TestOnReconnectListenerSupport, CoreAdminOperationTest, TestDynamicLoadingUrl, TestQueryUtils, TestHalfAndHalfDocValues, LeaderElectionTest, OrderedExecutorTest, TestFuzzyAnalyzedSuggestions, LukeRequestHandlerTest, RequiredFieldsTest, TestJsonFacetsWithNestedObjects, DirectUpdateHandlerTest, TestSimPolicyCloud, SolrCloudExampleTest, TestOmitPositions, CleanupOldIndexTest, HttpPartitionTest, TestCopyFieldCollectionResource, TestUniqueKeyFieldResource, TestClusterStateMutator, IndexBasedSpellCheckerTest, TestUseDocValuesAsStored2, FacetPivotSmallTest, TestSolrConfigHandlerConcurrent, TestQueryingOnDownCollection, TestRetrieveFieldsOptimizer, TestSolrCLIRunExample, TestFunctionQuery, BadCopyFieldTest, DistribJoinFromCollectionTest, SystemCollectionCompatTest, OverseerModifyCollectionTest, TestBlockCollapse, AuthToolTest, RankQParserPluginTest, ClusterStateUpdateTest, HdfsTlogReplayBufferedWhileIndexingTest]
   [junit4] Completed [176/934 (1!)] on J2 in 528.79s, 1 test, 1 failure <<< FAILURES!

[...truncated 53246 lines...]
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:652: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:507: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene/Lucene-Solr-NightlyTests-8.x/checkout/build.xml:494: Source checkout is dirty (unversioned/missing files) after running tests!!! Offending files:
* solr/contrib/analytics/src/test-files/solr/filestore
* solr/contrib/analytics/src/test-files/solr/userfiles
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/collection1/conf/dataimport.properties
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/filestore
* solr/contrib/dataimporthandler-extras/src/test-files/dihextras/solr/userfiles
* solr/contrib/dataimporthandler/src/test-files/dih/solr/filestore
* solr/contrib/dataimporthandler/src/test-files/dih/solr/userfiles
* solr/contrib/extraction/src/test-files/extraction/solr/filestore
* solr/contrib/extraction/src/test-files/extraction/solr/userfiles
* solr/contrib/langid/src/test-files/langid/solr/filestore
* solr/contrib/langid/src/test-files/langid/solr/userfiles
* solr/contrib/ltr/src/test-files/solr/filestore
* solr/contrib/ltr/src/test-files/solr/userfiles
* solr/contrib/prometheus-exporter/src/test-files/solr/filestore
* solr/contrib/prometheus-exporter/src/test-files/solr/userfiles
* solr/contrib/velocity/src/test-files/velocity/solr/filestore
* solr/contrib/velocity/src/test-files/velocity/solr/userfiles
* solr/core/src/test-files/solr/filestore
* solr/core/src/test-files/solr/userfiles
* solr/solrj/src/test-files/solrj/solr/multicore/filestore
* solr/solrj/src/test-files/solrj/solr/multicore/userfiles

Total time: 275 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any