You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2019/12/30 10:08:36 UTC
[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-12.0.1) - Build #
25230 - Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/25230/
Java: 64bit/jdk-12.0.1 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.TestCloudRecovery.corruptedLogTest
Error Message:
Captured an uncaught exception in thread: Thread[id=1197, name=replayUpdatesExecutor-439-thread-2, state=RUNNABLE, group=TGRP-TestCloudRecovery]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=1197, name=replayUpdatesExecutor-439-thread-2, state=RUNNABLE, group=TGRP-TestCloudRecovery]
at __randomizedtesting.SeedInfo.seed([96E0993E5E060112:1596C6CC887F0FB3]:0)
Caused by: java.lang.ArrayIndexOutOfBoundsException: Index 1 out of bounds for length 0
at __randomizedtesting.SeedInfo.seed([96E0993E5E060112]:0)
at java.base/java.util.ArrayList.add(ArrayList.java:486)
at java.base/java.util.ArrayList.add(ArrayList.java:498)
at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:115)
at org.apache.solr.update.UpdateLog$LogReplayer.lambda$execute$1(UpdateLog.java:2028)
at org.apache.solr.util.OrderedExecutor.lambda$execute$0(OrderedExecutor.java:69)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:210)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:835)
Build Log:
[...truncated 12944 lines...]
[junit4] Suite: org.apache.solr.cloud.TestCloudRecovery
[junit4] 2> 51704 INFO (SUITE-TestCloudRecovery-seed#[96E0993E5E060112]-worker) [ ] o.a.s.SolrTestCaseJ4 Created dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/data-dir-15-001
[junit4] 2> 51704 WARN (SUITE-TestCloudRecovery-seed#[96E0993E5E060112]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=5 numCloses=5
[junit4] 2> 51704 INFO (SUITE-TestCloudRecovery-seed#[96E0993E5E060112]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 51705 INFO (SUITE-TestCloudRecovery-seed#[96E0993E5E060112]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 51706 INFO (SUITE-TestCloudRecovery-seed#[96E0993E5E060112]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 51709 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.SolrTestCaseJ4 ###Starting corruptedLogTest
[junit4] 2> 51709 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/tempDir-001
[junit4] 2> 51710 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 51710 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 51710 INFO (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 51810 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.ZkTestServer start zk server on port:33117
[junit4] 2> 51810 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.ZkTestServer waitForServerUp: 127.0.0.1:33117
[junit4] 2> 51810 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:33117
[junit4] 2> 51810 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 33117
[junit4] 2> 51812 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 51814 INFO (zkConnectionManagerCallback-311-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 51814 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 51823 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 51825 INFO (zkConnectionManagerCallback-313-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 51825 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 51834 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 51841 INFO (zkConnectionManagerCallback-315-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 51842 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 51971 WARN (jetty-launcher-316-thread-1) [ ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
[junit4] 2> 51971 WARN (jetty-launcher-316-thread-2) [ ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
[junit4] 2> 51971 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 51971 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=0)
[junit4] 2> 51971 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 51971 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 0 try number 1 ...
[junit4] 2> 51971 INFO (jetty-launcher-316-thread-1) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 12.0.1+12
[junit4] 2> 51971 INFO (jetty-launcher-316-thread-2) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 12.0.1+12
[junit4] 2> 51972 INFO (jetty-launcher-316-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 51972 INFO (jetty-launcher-316-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 51972 INFO (jetty-launcher-316-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 51972 INFO (jetty-launcher-316-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 51972 INFO (jetty-launcher-316-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 51972 INFO (jetty-launcher-316-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 51972 INFO (jetty-launcher-316-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2bc6f255{/solr,null,AVAILABLE}
[junit4] 2> 51973 INFO (jetty-launcher-316-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1cc410aa{/solr,null,AVAILABLE}
[junit4] 2> 51974 INFO (jetty-launcher-316-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@15b96820{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:33363}
[junit4] 2> 51974 INFO (jetty-launcher-316-thread-1) [ ] o.e.j.s.Server Started @52005ms
[junit4] 2> 51974 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33363}
[junit4] 2> 51974 INFO (jetty-launcher-316-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7dbaef6b{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:43895}
[junit4] 2> 51974 ERROR (jetty-launcher-316-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 51974 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 51974 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 51974 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 51974 INFO (jetty-launcher-316-thread-2) [ ] o.e.j.s.Server Started @52006ms
[junit4] 2> 51974 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 51974 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-12-30T09:09:16.982891Z
[junit4] 2> 51974 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43895}
[junit4] 2> 51975 ERROR (jetty-launcher-316-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 51975 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 51975 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 51975 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 51975 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 51975 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-12-30T09:09:16.983219Z
[junit4] 2> 51976 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 51977 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 51981 INFO (zkConnectionManagerCallback-318-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 51981 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 51981 INFO (zkConnectionManagerCallback-320-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 51981 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 51983 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 51983 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 52136 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 52138 WARN (jetty-launcher-316-thread-2) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@3cc8c3ee[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 52138 WARN (jetty-launcher-316-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@3cc8c3ee[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 52141 WARN (jetty-launcher-316-thread-2) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@7fdbca18[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 52141 WARN (jetty-launcher-316-thread-2) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@7fdbca18[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 52142 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33117/solr
[junit4] 2> 52143 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 52144 INFO (zkConnectionManagerCallback-327-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 52144 INFO (jetty-launcher-316-thread-2) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 52288 INFO (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 52290 INFO (zkConnectionManagerCallback-330-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 52290 INFO (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 52290 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.h.c.HttpShardHandlerFactory Host whitelist initialized: WhitelistHostChecker [whitelistHosts=null, whitelistHostCheckingEnabled=true]
[junit4] 2> 52296 WARN (jetty-launcher-316-thread-1) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@22b64f72[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 52296 WARN (jetty-launcher-316-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@22b64f72[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 52301 WARN (jetty-launcher-316-thread-1) [ ] o.e.j.u.s.S.config Trusting all certificates configured for Client@6cfee8d8[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 52301 WARN (jetty-launcher-316-thread-1) [ ] o.e.j.u.s.S.config No Client EndPointIdentificationAlgorithm configured for Client@6cfee8d8[provider=null,keyStore=null,trustStore=null]
[junit4] 2> 52302 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33117/solr
[junit4] 2> 52302 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 52303 INFO (zkConnectionManagerCallback-338-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 52303 INFO (jetty-launcher-316-thread-1) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 52333 INFO (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43895_solr
[junit4] 2> 52333 INFO (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.c.Overseer Overseer (id=72071163034599430-127.0.0.1:43895_solr-n_0000000000) starting
[junit4] 2> 52336 INFO (OverseerStateUpdate-72071163034599430-127.0.0.1:43895_solr-n_0000000000) [n:127.0.0.1:43895_solr ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:43895_solr
[junit4] 2> 52337 INFO (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43895_solr
[junit4] 2> 52338 INFO (zkCallback-329-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 52338 WARN (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info
[junit4] 2> 52353 INFO (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 52371 INFO (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43895.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@104f88c9
[junit4] 2> 52376 INFO (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43895.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@104f88c9
[junit4] 2> 52377 INFO (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43895.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@104f88c9
[junit4] 2> 52378 INFO (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.p.PackageLoader /packages.json updated to version -1
[junit4] 2> 52379 INFO (jetty-launcher-316-thread-2) [n:127.0.0.1:43895_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/tempDir-001/node1/.
[junit4] 2> 52417 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 52425 INFO (zkConnectionManagerCallback-342-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 52425 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 52429 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 52432 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:33363_solr as DOWN
[junit4] 2> 52433 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 52433 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33363_solr
[junit4] 2> 52434 INFO (zkCallback-329-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 52434 WARN (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.c.CoreContainer Not all security plugins configured! authentication=disabled authorization=disabled. Solr is only as secure as you make it. Consider configuring authentication/authorization before exposing Solr to users internal or external. See https://s.apache.org/solrsecurity for more info
[junit4] 2> 52442 INFO (zkCallback-341-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 52454 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 52471 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33363.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@104f88c9
[junit4] 2> 52477 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33363.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@104f88c9
[junit4] 2> 52477 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33363.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@104f88c9
[junit4] 2> 52478 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.p.PackageLoader /packages.json updated to version -1
[junit4] 2> 52479 INFO (jetty-launcher-316-thread-1) [n:127.0.0.1:33363_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/tempDir-001/node2/.
[junit4] 2> 52500 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.MiniSolrCloudCluster waitForAllNodes: numServers=2
[junit4] 2> 52506 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 52509 INFO (zkConnectionManagerCallback-351-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 52509 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.c.ConnectionManager Client is connected to ZooKeeper
[junit4] 2> 52511 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 52512 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33117/solr ready
[junit4] 2> 52533 INFO (qtp1514374947-985) [n:127.0.0.1:33363_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&collection.configName=config&maxShardsPerNode=2&name=collection1&nrtReplicas=2&action=CREATE&numShards=2&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 52535 INFO (OverseerThreadFactory-402-thread-1-processing-n:127.0.0.1:43895_solr) [n:127.0.0.1:43895_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 52640 INFO (OverseerStateUpdate-72071163034599430-127.0.0.1:43895_solr-n_0000000000) [n:127.0.0.1:43895_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33363/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 52641 INFO (OverseerStateUpdate-72071163034599430-127.0.0.1:43895_solr-n_0000000000) [n:127.0.0.1:43895_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43895/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 52642 INFO (OverseerStateUpdate-72071163034599430-127.0.0.1:43895_solr-n_0000000000) [n:127.0.0.1:43895_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33363/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 52644 INFO (OverseerStateUpdate-72071163034599430-127.0.0.1:43895_solr-n_0000000000) [n:127.0.0.1:43895_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43895/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 52870 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr x:collection1_shard2_replica_n4 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=config&newCollection=true&name=collection1_shard2_replica_n4&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 52879 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr x:collection1_shard2_replica_n6 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=config&newCollection=true&name=collection1_shard2_replica_n6&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 52879 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr x:collection1_shard2_replica_n6 ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 52880 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr x:collection1_shard1_replica_n1 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 52884 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr x:collection1_shard1_replica_n2 ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=config&newCollection=true&name=collection1_shard1_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 53883 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 53900 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.s.IndexSchema [collection1_shard2_replica_n4] Schema name=minimal
[junit4] 2> 53901 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 53901 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n4' using configuration from collection collection1, trusted=true
[junit4] 2> 53902 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33363.solr.core.collection1.shard2.replica_n4' (registry 'solr.core.collection1.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@104f88c9
[junit4] 2> 53903 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 53903 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 53903 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 9.0.0
[junit4] 2> 53907 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.SolrCore [[collection1_shard2_replica_n4] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/tempDir-001/node2/collection1_shard2_replica_n4], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/tempDir-001/node2/./collection1_shard2_replica_n4/data/]
[junit4] 2> 53919 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.s.IndexSchema [collection1_shard2_replica_n6] Schema name=minimal
[junit4] 2> 53920 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=minimal
[junit4] 2> 53921 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 53921 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n6' using configuration from collection collection1, trusted=true
[junit4] 2> 53922 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 53922 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.s.IndexSchema [collection1_shard1_replica_n2] Schema name=minimal
[junit4] 2> 53922 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43895.solr.core.collection1.shard2.replica_n6' (registry 'solr.core.collection1.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@104f88c9
[junit4] 2> 53923 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.c.SolrCore [[collection1_shard2_replica_n6] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/tempDir-001/node1/collection1_shard2_replica_n6], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/tempDir-001/node1/./collection1_shard2_replica_n6/data/]
[junit4] 2> 53924 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 53924 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from collection collection1, trusted=true
[junit4] 2> 53924 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33363.solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@104f88c9
[junit4] 2> 53925 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/tempDir-001/node2/collection1_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/tempDir-001/node2/./collection1_shard1_replica_n1/data/]
[junit4] 2> 53928 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n2' using configuration from collection collection1, trusted=true
[junit4] 2> 53929 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43895.solr.core.collection1.shard1.replica_n2' (registry 'solr.core.collection1.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@104f88c9
[junit4] 2> 53929 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.SolrCore [[collection1_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/tempDir-001/node1/collection1_shard1_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestCloudRecovery_96E0993E5E060112-001/tempDir-001/node1/./collection1_shard1_replica_n2/data/]
[junit4] 2> 53991 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 53991 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 53996 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 53996 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 53996 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 53996 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 53998 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 53998 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 54000 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.s.SolrIndexSearcher Opening [Searcher@197b2ffd[collection1_shard2_replica_n4] main]
[junit4] 2> 54001 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 54002 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 54002 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.s.SolrIndexSearcher Opening [Searcher@7ace7bee[collection1_shard2_replica_n6] main]
[junit4] 2> 54002 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 54002 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1654335166490869760
[junit4] 2> 54003 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 54004 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 54004 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 54004 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1654335166492966912
[junit4] 2> 54005 INFO (searcherExecutor-411-thread-1-processing-n:127.0.0.1:33363_solr x:collection1_shard2_replica_n4 c:collection1 s:shard2 r:core_node7) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.SolrCore [collection1_shard2_replica_n4] Registered new searcher Searcher@197b2ffd[collection1_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 54021 INFO (searcherExecutor-412-thread-1-processing-n:127.0.0.1:43895_solr x:collection1_shard2_replica_n6 c:collection1 s:shard2 r:core_node8) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.c.SolrCore [collection1_shard2_replica_n6] Registered new searcher Searcher@7ace7bee[collection1_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 54021 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 54021 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 54022 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 54022 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 54022 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node7=0}, version=0}
[junit4] 2> 54022 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2
[junit4] 2> 54024 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node7=0, core_node8=0}, version=1}
[junit4] 2> 54024 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard2
[junit4] 2> 54025 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.SolrIndexSearcher Opening [Searcher@48de9367[collection1_shard1_replica_n1] main]
[junit4] 2> 54026 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 54026 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 54027 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 54027 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1654335166517084160
[junit4] 2> 54027 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 54027 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 54027 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:33363/solr/collection1_shard2_replica_n4/
[junit4] 2> 54029 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.PeerSync PeerSync: core=collection1_shard2_replica_n4 url=https://127.0.0.1:33363/solr START replicas=[https://127.0.0.1:43895/solr/collection1_shard2_replica_n6/] nUpdates=100
[junit4] 2> 54029 INFO (searcherExecutor-413-thread-1-processing-n:127.0.0.1:33363_solr x:collection1_shard1_replica_n1 c:collection1 s:shard1 r:core_node3) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [collection1_shard1_replica_n1] Registered new searcher Searcher@48de9367[collection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 54034 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.PeerSync PeerSync: core=collection1_shard2_replica_n4 url=https://127.0.0.1:33363/solr DONE. We have no versions. sync failed.
[junit4] 2> 54035 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=0}, version=0}
[junit4] 2> 54036 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
[junit4] 2> 54039 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 54049 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 54049 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 54050 INFO (qtp843058536-986) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.c.S.Request [collection1_shard2_replica_n6] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 54051 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 54051 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 54051 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 54052 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 54052 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard2/leader after winning as /collections/collection1/leader_elect/shard2/election/72071163034599432-core_node7-n_0000000000
[junit4] 2> 54053 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:33363/solr/collection1_shard2_replica_n4/ shard2
[junit4] 2> 54053 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.s.SolrIndexSearcher Opening [Searcher@419094b6[collection1_shard1_replica_n2] main]
[junit4] 2> 54055 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 54055 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 54055 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 54055 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1654335166546444288
[junit4] 2> 54057 INFO (searcherExecutor-414-thread-1-processing-n:127.0.0.1:43895_solr x:collection1_shard1_replica_n2 c:collection1 s:shard1 r:core_node5) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.SolrCore [collection1_shard1_replica_n2] Registered new searcher Searcher@419094b6[collection1_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 54059 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=0, core_node5=0}, version=1}
[junit4] 2> 54059 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.ShardLeaderElectionContextBase make sure parent is created /collections/collection1/leaders/shard1
[junit4] 2> 54155 INFO (zkCallback-341-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54155 INFO (zkCallback-341-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54155 INFO (zkCallback-341-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54156 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 54158 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=config&newCollection=true&name=collection1_shard2_replica_n4&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1288
[junit4] 2> 54263 INFO (zkCallback-341-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54263 INFO (zkCallback-341-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54263 INFO (zkCallback-341-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54265 INFO (zkCallback-341-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54540 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 54540 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 54540 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:33363/solr/collection1_shard1_replica_n1/
[junit4] 2> 54540 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n1 url=https://127.0.0.1:33363/solr START replicas=[https://127.0.0.1:43895/solr/collection1_shard1_replica_n2/] nUpdates=100
[junit4] 2> 54540 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n1 url=https://127.0.0.1:33363/solr DONE. We have no versions. sync failed.
[junit4] 2> 54542 INFO (qtp843058536-1059) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.S.Request [collection1_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 54543 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 54543 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 54543 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/collection1/leaders/shard1/leader after winning as /collections/collection1/leader_elect/shard1/election/72071163034599432-core_node3-n_0000000000
[junit4] 2> 54544 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:33363/solr/collection1_shard1_replica_n1/ shard1
[junit4] 2> 54645 INFO (zkCallback-341-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54645 INFO (zkCallback-341-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54645 INFO (zkCallback-341-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54645 INFO (zkCallback-341-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54646 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 54650 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1770
[junit4] 2> 54748 INFO (zkCallback-341-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54748 INFO (zkCallback-341-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54748 INFO (zkCallback-341-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 54748 INFO (zkCallback-341-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55030 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=config&newCollection=true&name=collection1_shard2_replica_n6&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2151
[junit4] 2> 55062 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=config&newCollection=true&name=collection1_shard1_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2179
[junit4] 2> 55065 INFO (qtp1514374947-985) [n:127.0.0.1:33363_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 45 seconds. Check all shard replicas
[junit4] 2> 55164 INFO (zkCallback-341-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55164 INFO (zkCallback-329-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55164 INFO (zkCallback-341-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55164 INFO (zkCallback-329-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55164 INFO (zkCallback-341-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55164 INFO (zkCallback-341-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55164 INFO (qtp1514374947-985) [n:127.0.0.1:33363_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&collection.configName=config&maxShardsPerNode=2&name=collection1&nrtReplicas=2&action=CREATE&numShards=2&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=2631
[junit4] 2> 55165 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.MiniSolrCloudCluster waitForActiveCollection: collection1
[junit4] 2> 55194 INFO (qtp1514374947-1047) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard1 to Terms{values={core_node3=1, core_node5=1}, version=2}
[junit4] 2> 55250 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{add=[1000 (1654335167733432320)]} 0 3
[junit4] 2> 55252 INFO (qtp1514374947-1047) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1000 (1654335167733432320)]} 0 73
[junit4] 2> 55261 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node7=1, core_node8=1}, version=2}
[junit4] 2> 55264 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n6] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard2_replica_n4/&wt=javabin&version=2}{add=[1001 (1654335167806832640)]} 0 2
[junit4] 2> 55265 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n4] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1001 (1654335167806832640)]} 0 9
[junit4] 2> 55270 INFO (qtp843058536-986) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{add=[0 (1654335167817318400)]} 0 1
[junit4] 2> 55271 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0 (1654335167817318400)]} 0 4
[junit4] 2> 55276 INFO (qtp843058536-1059) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{add=[1 (1654335167824658432)]} 0 0
[junit4] 2> 55277 INFO (qtp1514374947-985) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1 (1654335167824658432)]} 0 3
[junit4] 2> 55289 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n6] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard2_replica_n4/&wt=javabin&version=2}{add=[2 (1654335167829901312)]} 0 5
[junit4] 2> 55290 INFO (qtp1514374947-1047) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n4] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2 (1654335167829901312)]} 0 11
[junit4] 2> 55296 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n6] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard2_replica_n4/&wt=javabin&version=2}{add=[3 (1654335167843532800)]} 0 0
[junit4] 2> 55296 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n4] webapp=/solr path=/update params={wt=javabin&version=2}{add=[3 (1654335167843532800)]} 0 4
[junit4] 2> 55302 INFO (qtp843058536-986) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{add=[4 (1654335167850872832)]} 0 1
[junit4] 2> 55303 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[4 (1654335167850872832)]} 0 4
[junit4] 2> 55310 INFO (qtp843058536-1059) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n6] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard2_replica_n4/&wt=javabin&version=2}{add=[5 (1654335167859261440)]} 0 0
[junit4] 2> 55311 INFO (qtp1514374947-985) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n4] webapp=/solr path=/update params={wt=javabin&version=2}{add=[5 (1654335167859261440)]} 0 6
[junit4] 2> 55317 INFO (qtp843058536-984) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n6] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard2_replica_n4/&wt=javabin&version=2}{add=[6 (1654335167866601472)]} 0 0
[junit4] 2> 55317 INFO (qtp1514374947-1047) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n4] webapp=/solr path=/update params={wt=javabin&version=2}{add=[6 (1654335167866601472)]} 0 4
[junit4] 2> 55323 INFO (qtp843058536-988) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n6] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard2_replica_n4/&wt=javabin&version=2}{add=[7 (1654335167872892928)]} 0 0
[junit4] 2> 55323 INFO (qtp1514374947-987) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n4] webapp=/solr path=/update params={wt=javabin&version=2}{add=[7 (1654335167872892928)]} 0 3
[junit4] 2> 55329 INFO (qtp843058536-986) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{add=[8 (1654335167879184384)]} 0 1
[junit4] 2> 55329 INFO (qtp1514374947-983) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[8 (1654335167879184384)]} 0 4
[junit4] 2> 55333 INFO (qtp843058536-1059) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n6] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:33363/solr/collection1_shard2_replica_n4/&wt=javabin&version=2}{add=[9 (1654335167884427264)]} 0 0
[junit4] 2> 55333 INFO (qtp1514374947-985) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n4] webapp=/solr path=/update params={wt=javabin&version=2}{add=[9 (1654335167884427264)]} 0 2
[junit4] 2> 55386 INFO (qtp843058536-986) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.c.S.Request [collection1_shard2_replica_n6] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=collection1:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=https://127.0.0.1:43895/solr/collection1_shard2_replica_n6/|https://127.0.0.1:33363/solr/collection1_shard2_replica_n4/&rows=10&version=2&q=*:*&omitHeader=false&NOW=1577696960343&isShard=true&wt=javabin} hits=0 status=0 QTime=0
[junit4] 2> 55386 INFO (qtp843058536-1059) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.S.Request [collection1_shard1_replica_n2] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=collection1:7&fl=id&fl=score&shards.purpose=16388&start=0&fsv=true&shard.url=https://127.0.0.1:43895/solr/collection1_shard1_replica_n2/|https://127.0.0.1:33363/solr/collection1_shard1_replica_n1/&rows=10&version=2&q=*:*&omitHeader=false&NOW=1577696960343&isShard=true&wt=javabin} hits=0 status=0 QTime=0
[junit4] 2> 55388 INFO (qtp1514374947-1047) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.S.Request [collection1_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=collection1:7&wt=javabin&version=2} hits=0 status=0 QTime=52
[junit4] 2> 55393 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=630437939
[junit4] 2> 55393 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:43895_solr
[junit4] 2> 55394 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.c.ZkController Publish this node as DOWN...
[junit4] 2> 55394 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.c.ZkController Publish node=127.0.0.1:43895_solr as DOWN
[junit4] 2> 55395 INFO (ChaosMonkey-431-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1032186823
[junit4] 2> 55395 INFO (ChaosMonkey-431-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33363_solr
[junit4] 2> 55395 INFO (ChaosMonkey-431-thread-2) [ ] o.a.s.c.ZkController Publish this node as DOWN...
[junit4] 2> 55396 INFO (ChaosMonkey-431-thread-2) [ ] o.a.s.c.ZkController Publish node=127.0.0.1:33363_solr as DOWN
[junit4] 2> 55425 INFO (coreCloseExecutor-432-thread-1) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.c.SolrCore [collection1_shard2_replica_n6] CLOSING SolrCore org.apache.solr.core.SolrCore@1d4e44fa
[junit4] 2> 55425 INFO (coreCloseExecutor-432-thread-1) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard2.replica_n6, tag=SolrCore@1d4e44fa
[junit4] 2> 55425 INFO (coreCloseExecutor-432-thread-1) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1d6cead4: rootName = solr_43895, domain = solr.core.collection1.shard2.replica_n6, service url = null, agent id = null] for registry solr.core.collection1.shard2.replica_n6 / com.codahale.metrics.MetricRegistry@93e2062
[junit4] 2> 55436 INFO (coreCloseExecutor-432-thread-1) [n:127.0.0.1:43895_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n6 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=SolrCore@1d4e44fa
[junit4] 2> 55458 INFO (coreCloseExecutor-432-thread-2) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.c.SolrCore [collection1_shard1_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@635981ea
[junit4] 2> 55458 INFO (coreCloseExecutor-432-thread-2) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n2, tag=SolrCore@635981ea
[junit4] 2> 55458 INFO (coreCloseExecutor-432-thread-2) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2c0ffc1f: rootName = solr_43895, domain = solr.core.collection1.shard1.replica_n2, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_n2 / com.codahale.metrics.MetricRegistry@7e5714b1
[junit4] 2> 55473 INFO (coreCloseExecutor-433-thread-1) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.c.SolrCore [collection1_shard2_replica_n4] CLOSING SolrCore org.apache.solr.core.SolrCore@1d714b5f
[junit4] 2> 55473 INFO (coreCloseExecutor-433-thread-2) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.c.SolrCore [collection1_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@7e6be2b9
[junit4] 2> 55473 INFO (coreCloseExecutor-433-thread-2) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n1, tag=SolrCore@7e6be2b9
[junit4] 2> 55473 INFO (coreCloseExecutor-433-thread-2) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6c2b29c1: rootName = solr_33363, domain = solr.core.collection1.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@2019a4b2
[junit4] 2> 55486 INFO (coreCloseExecutor-433-thread-1) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard2.replica_n4, tag=SolrCore@1d714b5f
[junit4] 2> 55486 INFO (coreCloseExecutor-433-thread-1) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@464fe2b: rootName = solr_33363, domain = solr.core.collection1.shard2.replica_n4, service url = null, agent id = null] for registry solr.core.collection1.shard2.replica_n4 / com.codahale.metrics.MetricRegistry@4e27194e
[junit4] 2> 55490 INFO (coreCloseExecutor-432-thread-2) [n:127.0.0.1:43895_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=SolrCore@635981ea
[junit4] 2> 55493 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 55493 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@47ce0f71: rootName = solr_43895, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6b014344
[junit4] 2> 55497 INFO (coreCloseExecutor-433-thread-1) [n:127.0.0.1:33363_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_n4 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=SolrCore@1d714b5f
[junit4] 2> 55498 INFO (zkCallback-341-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55498 INFO (zkCallback-341-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55498 INFO (zkCallback-341-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55498 INFO (zkCallback-329-thread-3) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55498 INFO (zkCallback-341-thread-4) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55498 INFO (zkCallback-329-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 55499 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 55500 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@666bcad3: rootName = solr_43895, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@62ed386
[junit4] 2> 55501 INFO (coreCloseExecutor-433-thread-2) [n:127.0.0.1:33363_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=SolrCore@7e6be2b9
[junit4] 2> 55511 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 55512 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3f514d23: rootName = solr_43895, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1347a4c9
[junit4] 2> 55512 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 55517 INFO (closeThreadPool-362-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72071163034599430-127.0.0.1:43895_solr-n_0000000000) closing
[junit4] 2> 55517 INFO (OverseerStateUpdate-72071163034599430-127.0.0.1:43895_solr-n_0000000000) [n:127.0.0.1:43895_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:43895_solr
[junit4] 2> 55517 INFO (OverseerAutoScalingTriggerThread-72071163034599430-127.0.0.1:43895_solr-n_0000000000) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 55528 INFO (closeThreadPool-362-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72071163034599430-127.0.0.1:43895_solr-n_0000000000) closing
[junit4] 2> 55528 INFO (zkCallback-350-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 55530 INFO (zkCallback-341-thread-4) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 55534 INFO (ChaosMonkey-431-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 55534 INFO (ChaosMonkey-431-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@649d1bf9: rootName = solr_33363, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@704953db
[junit4] 2> 55540 INFO (ChaosMonkey-431-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 55540 INFO (ChaosMonkey-431-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@20e9b81e: rootName = solr_33363, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@62ed386
[junit4] 2> 55543 INFO (ChaosMonkey-431-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 55543 INFO (ChaosMonkey-431-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@353f60d6: rootName = solr_33363, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1347a4c9
[junit4] 2> 55543 INFO (ChaosMonkey-431-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 55558 INFO (zkCallback-350-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 55629 INFO (ChaosMonkey-431-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72071163034599430-127.0.0.1:43895_solr-n_0000000000) closing
[junit4] 2> 55631 INFO (ChaosMonkey-431-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7dbaef6b{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:0}
[junit4] 2> 55631 INFO (ChaosMonkey-431-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1cc410aa{/solr,null,UNAVAILABLE}
[junit4] 2> 55631 INFO (ChaosMonkey-431-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 55731 INFO (ChaosMonkey-431-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@15b96820{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:0}
[junit4] 2> 55731 INFO (ChaosMonkey-431-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2bc6f255{/solr,null,UNAVAILABLE}
[junit4] 2> 55731 INFO (ChaosMonkey-431-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 55732 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.MiniSolrCloudCluster waitForJettyToStop: 43895
[junit4] 2> 55732 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.MiniSolrCloudCluster waitForJettyToStop: 33363
[junit4] 2> 55745 INFO (ChaosMonkey-436-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=43895)
[junit4] 2> 55750 WARN (ChaosMonkey-436-thread-1) [ ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
[junit4] 2> 55750 INFO (ChaosMonkey-436-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 43895 try number 1 ...
[junit4] 2> 55750 INFO (ChaosMonkey-436-thread-1) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 12.0.1+12
[junit4] 2> 55753 INFO (ChaosMonkey-436-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Start Jetty (configured port=0, binding port=33363)
[junit4] 2> 55757 WARN (ChaosMonkey-436-thread-2) [ ] o.e.j.s.h.g.GzipHandler minGzipSize of 0 is inefficient for short content, break even is size 23
[junit4] 2> 55757 INFO (ChaosMonkey-436-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Trying to start Jetty on port 33363 try number 1 ...
[junit4] 2> 55757 INFO (ChaosMonkey-436-thread-2) [ ] o.e.j.s.Server jetty-9.4.24.v20191120; built: 2019-11-20T21:37:49.771Z; git: 363d5f2df3a8a28de40604320230664b9c793c16; jvm 12.0.1+12
[junit4] 2> 55821 INFO (ChaosMonkey-436-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 55821 INFO (ChaosMonkey-436-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 55821 INFO (ChaosMonkey-436-thread-2) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 55829 INFO (ChaosMonkey-436-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 55829 INFO (ChaosMonkey-436-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 55829 INFO (ChaosMonkey-436-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 55837 INFO (ChaosMonkey-436-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@538157d1{/solr,null,AVAILABLE}
[junit4] 2> 55837 INFO (ChaosMonkey-436-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@298dacfc{/solr,null,AVAILABLE}
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@37539d3f{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:33363}
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@268f256{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:43895}
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-2) [ ] o.e.j.s.Server Started @55872ms
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-1) [ ] o.e.j.s.Server Started @55872ms
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33363}
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43895}
[junit4] 2> 55841 ERROR (ChaosMonkey-436-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 55841 ERROR (ChaosMonkey-436-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 9.0.0
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 55841 INFO (ChaosMonkey-436-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 55842 INFO (ChaosMonkey-436-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 55842 INFO (ChaosMonkey-436-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 55842 INFO (ChaosMonkey-436-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-12-30T09:09:20.850017Z
[junit4] 2> 55842 INFO (ChaosMonkey-436-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2019-12-30T09:09:20.850016Z
[junit4] 2> 55859 INFO (ChaosMonkey-436-thread-2) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKeeper
[junit4] 2> 55859 INFO (ChaosMonkey-436-thread-1) [ ] o.a.s.c.c.ConnectionManager Waiting for client to connect to ZooKee
[...truncated too long message...]
rs.SolrJmxReporter@1c94bc46: rootName = solr_34397, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5645d5d4
[junit4] 2> 67638 INFO (jetty-closer-499-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 67638 INFO (jetty-closer-499-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6a72796c: rootName = solr_34397, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@62ed386
[junit4] 2> 67640 INFO (jetty-closer-499-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 67640 INFO (jetty-closer-499-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4d5b86be: rootName = solr_34397, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1347a4c9
[junit4] 2> 67640 INFO (jetty-closer-499-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 67640 INFO (closeThreadPool-506-thread-2) [ ] o.a.s.c.Overseer Overseer (id=72071163581825037-127.0.0.1:34397_solr-n_0000000002) closing
[junit4] 2> 67640 INFO (OverseerStateUpdate-72071163581825037-127.0.0.1:34397_solr-n_0000000002) [n:127.0.0.1:34397_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34397_solr
[junit4] 2> 67641 INFO (OverseerAutoScalingTriggerThread-72071163581825037-127.0.0.1:34397_solr-n_0000000002) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 67642 INFO (closeThreadPool-506-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72071163581825037-127.0.0.1:34397_solr-n_0000000002) closing
[junit4] 2> 67642 INFO (coreCloseExecutor-575-thread-2) [n:127.0.0.1:45457_solr c:collection1 s:shard1 r:core_node5 x:collection1_shard1_replica_n2 ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=SolrCore@42c8fcde
[junit4] 2> 67643 INFO (zkCallback-489-thread-4) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 67644 INFO (jetty-closer-499-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 67644 INFO (jetty-closer-499-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@60adbde5: rootName = solr_45457, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@50022f6a
[junit4] 2> 67649 INFO (jetty-closer-499-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 67649 INFO (jetty-closer-499-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@161ab36c: rootName = solr_45457, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@62ed386
[junit4] 2> 67650 INFO (jetty-closer-499-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 67650 INFO (jetty-closer-499-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@601219e6: rootName = solr_45457, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1347a4c9
[junit4] 2> 67650 INFO (jetty-closer-499-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 67743 INFO (jetty-closer-499-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72071163581825037-127.0.0.1:34397_solr-n_0000000002) closing
[junit4] 2> 67746 INFO (jetty-closer-499-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@555ac762{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:34397}
[junit4] 2> 67746 INFO (jetty-closer-499-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3eb2fb79{/solr,null,UNAVAILABLE}
[junit4] 2> 67747 INFO (jetty-closer-499-thread-1) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 67844 INFO (jetty-closer-499-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5090346e{ssl,[ssl, alpn, http/1.1, h2]}{127.0.0.1:45457}
[junit4] 2> 67844 INFO (jetty-closer-499-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2fc861a5{/solr,null,UNAVAILABLE}
[junit4] 2> 67844 INFO (jetty-closer-499-thread-2) [ ] o.e.j.s.session node0 Stopped scavenging
[junit4] 2> 67845 INFO (TEST-TestCloudRecovery.leaderRecoverFromLogOnStartupTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.ZkTestServer Shutting down ZkTestServer.
[junit4] 2> 68048 WARN (ZkTestServer Run Thread) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 9 /solr/collections/collection1/terms/shard2
[junit4] 2> 9 /solr/collections/collection1/terms/shard1
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 4 /solr/packages.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 4 /solr/collections/collection1/collectionprops.json
[junit4] 2> 4 /solr/configs/config
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 57 /solr/collections/collection1/state.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 2 /solr/autoscaling.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 48 /solr/overseer/queue
[junit4] 2> 17 /solr/live_nodes
[junit4] 2> 8 /solr/collections
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/.scheduled_maintenance
[junit4] 2> 2 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 68049 INFO (TEST-TestCloudRecovery.leaderRecoverFromLogOnStartupTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.ZkTestServer waitForServerDown: 127.0.0.1:34069
[junit4] 2> 68049 INFO (TEST-TestCloudRecovery.leaderRecoverFromLogOnStartupTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.ZkTestServer parse host and port list: 127.0.0.1:34069
[junit4] 2> 68049 INFO (TEST-TestCloudRecovery.leaderRecoverFromLogOnStartupTest-seed#[96E0993E5E060112]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1 34069
[junit4] 2> 68050 INFO (TEST-TestCloudRecovery.leaderRecoverFromLogOnStartupTest-seed#[96E0993E5E060112]) [ ] o.a.s.SolrTestCaseJ4 ###Ending leaderRecoverFromLogOnStartupTest
[junit4] 2> 68051 INFO (SUITE-TestCloudRecovery-seed#[96E0993E5E060112]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Dec 30, 2019 9:09:33 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene84): {_root_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene84)), id=PostingsFormat(name=MockRandom)}, docValues:{_version_=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=1015, maxMBSortInHeap=6.825800904088554, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3dd61990), locale=es-UY, timezone=America/Guatemala
[junit4] 2> NOTE: Linux 5.0.0-37-generic amd64/AdoptOpenJDK 12.0.1 (64-bit)/cpus=8,threads=1,free=162028632,total=365953024
[junit4] 2> NOTE: All tests run in this JVM: [PeerSyncWithLeaderAndIndexFingerprintCachingTest, ShardRoutingCustomTest, SolrIndexMetricsTest, TestManagedSynonymFilterFactory, SchemaApiFailureTest, CircularListTest, TestUnInvertedFieldException, BasicAuthOnSingleNodeTest, InfoHandlerTest, DirectSolrConnectionTest, HdfsTlogReplayBufferedWhileIndexingTest, ConfigSetsAPITest, ClusterStateTest, TestTolerantSearch, TestCloudRecovery]
[junit4] Completed [25/894 (1!)] on J2 in 16.79s, 2 tests, 1 error <<< FAILURES!
[...truncated 47901 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/25230/consoleText
[repro] Revision: ca6bd364fb0a5a454b2f92d8a78f35a8b87339f8
[repro] Ant options: "-Dargs=-XX:-UseCompressedOops -XX:+UseG1GC"
[repro] Repro line: ant test -Dtestcase=TestCloudRecovery -Dtests.method=corruptedLogTest -Dtests.seed=96E0993E5E060112 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es-UY -Dtests.timezone=America/Guatemala -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[repro] JUnit rest result XML files will be moved to: ./repro-reports
[repro] ant clean
[...truncated 6 lines...]
[repro] Test suites by module:
[repro] solr/core
[repro] TestCloudRecovery
[repro] ant compile-test
[...truncated 3333 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestCloudRecovery" -Dtests.showOutput=onerror "-Dargs=-XX:-UseCompressedOops -XX:+UseG1GC" -Dtests.seed=96E0993E5E060112 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es-UY -Dtests.timezone=America/Guatemala -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[...truncated 80 lines...]
[repro] Failures w/original seeds:
[repro] 0/5 failed: org.apache.solr.cloud.TestCloudRecovery
[repro] Exiting with code 0
[...truncated 43 lines...]