You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/06/24 23:05:48 UTC

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk1.8.0_172) - Build # 7378 - Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7378/
Java: 64bit/jdk1.8.0_172 -XX:+UseCompressedOops -XX:+UseParallelGC

3 tests failed.
FAILED:  org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir

Error Message:
Captured an uncaught exception in thread: Thread[id=29262, name=cdcr-replicator-12382-thread-1, state=RUNNABLE, group=TGRP-CdcrBidirectionalTest]

Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=29262, name=cdcr-replicator-12382-thread-1, state=RUNNABLE, group=TGRP-CdcrBidirectionalTest]
Caused by: java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([E3B508AB83D18620]:0)
	at org.apache.solr.update.CdcrUpdateLog$CdcrLogReader.forwardSeek(CdcrUpdateLog.java:611)
	at org.apache.solr.handler.CdcrReplicator.run(CdcrReplicator.java:125)
	at org.apache.solr.handler.CdcrReplicatorScheduler.lambda$null$0(CdcrReplicatorScheduler.java:81)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  junit.framework.TestSuite.org.apache.solr.schema.TestUseDocValuesAsStored2

Error Message:
Could not remove the following files (in the order of attempts):    C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003\cores\core: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003\cores\core    C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003\cores: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003\cores    C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003 

Stack Trace:
java.io.IOException: Could not remove the following files (in the order of attempts):
   C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003\cores\core: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003\cores\core
   C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003\cores: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003\cores
   C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.schema.TestUseDocValuesAsStored2_E3B508AB83D18620-001\tempDir-003

	at __randomizedtesting.SeedInfo.seed([E3B508AB83D18620]:0)
	at org.apache.lucene.util.IOUtils.rm(IOUtils.java:318)
	at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
	at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)


FAILED:  org.apache.solr.common.util.TestTimeSource.testEpochTime

Error Message:
NanoTimeSource time diff=424600

Stack Trace:
java.lang.AssertionError: NanoTimeSource time diff=424600
	at __randomizedtesting.SeedInfo.seed([51F3195FCA7E1898:699F6A7A5EAEBADE]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.common.util.TestTimeSource.doTestEpochTime(TestTimeSource.java:52)
	at org.apache.solr.common.util.TestTimeSource.testEpochTime(TestTimeSource.java:31)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.lang.Thread.run(Thread.java:748)




Build Log:
[...truncated 14591 lines...]
   [junit4] Suite: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest
   [junit4]   2> 3116698 INFO  (SUITE-CdcrBidirectionalTest-seed#[E3B508AB83D18620]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\init-core-data-001
   [junit4]   2> 3116700 INFO  (SUITE-CdcrBidirectionalTest-seed#[E3B508AB83D18620]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 3116701 INFO  (SUITE-CdcrBidirectionalTest-seed#[E3B508AB83D18620]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 3116703 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testBiDir
   [junit4]   2> 3116704 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster2-001
   [junit4]   2> 3116704 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3116704 INFO  (Thread-5181) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3116704 INFO  (Thread-5181) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3116708 ERROR (Thread-5181) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 3116805 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.ZkTestServer start zk server on port:51494
   [junit4]   2> 3116808 INFO  (zkConnectionManagerCallback-8195-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3116813 INFO  (jetty-launcher-8192-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 3116813 INFO  (jetty-launcher-8192-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3116813 INFO  (jetty-launcher-8192-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3116814 INFO  (jetty-launcher-8192-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 3116814 INFO  (jetty-launcher-8192-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@59b7472{/solr,null,AVAILABLE}
   [junit4]   2> 3116816 INFO  (jetty-launcher-8192-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@51917e40{HTTP/1.1,[http/1.1]}{127.0.0.1:51498}
   [junit4]   2> 3116816 INFO  (jetty-launcher-8192-thread-1) [    ] o.e.j.s.Server Started @3116852ms
   [junit4]   2> 3116816 INFO  (jetty-launcher-8192-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51498}
   [junit4]   2> 3116816 ERROR (jetty-launcher-8192-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3116816 INFO  (jetty-launcher-8192-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 3116816 INFO  (jetty-launcher-8192-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 3116816 INFO  (jetty-launcher-8192-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3116816 INFO  (jetty-launcher-8192-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3116816 INFO  (jetty-launcher-8192-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-24T22:18:52.502Z
   [junit4]   2> 3116820 INFO  (zkConnectionManagerCallback-8197-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3116820 INFO  (jetty-launcher-8192-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3116882 INFO  (jetty-launcher-8192-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51494/solr
   [junit4]   2> 3116884 INFO  (zkConnectionManagerCallback-8201-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3116888 INFO  (zkConnectionManagerCallback-8203-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3116982 INFO  (jetty-launcher-8192-thread-1) [n:127.0.0.1:51498_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3116982 INFO  (jetty-launcher-8192-thread-1) [n:127.0.0.1:51498_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51498_solr
   [junit4]   2> 3116983 INFO  (jetty-launcher-8192-thread-1) [n:127.0.0.1:51498_solr    ] o.a.s.c.Overseer Overseer (id=72120976566452227-127.0.0.1:51498_solr-n_0000000000) starting
   [junit4]   2> 3116991 INFO  (zkConnectionManagerCallback-8210-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3116993 INFO  (jetty-launcher-8192-thread-1) [n:127.0.0.1:51498_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51494/solr ready
   [junit4]   2> 3116994 INFO  (jetty-launcher-8192-thread-1) [n:127.0.0.1:51498_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51498_solr
   [junit4]   2> 3116997 INFO  (zkCallback-8202-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3116997 INFO  (zkCallback-8209-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3117043 INFO  (jetty-launcher-8192-thread-1) [n:127.0.0.1:51498_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 3117090 INFO  (jetty-launcher-8192-thread-1) [n:127.0.0.1:51498_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51498.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29395f9d
   [junit4]   2> 3117095 INFO  (jetty-launcher-8192-thread-1) [n:127.0.0.1:51498_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51498.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29395f9d
   [junit4]   2> 3117095 INFO  (jetty-launcher-8192-thread-1) [n:127.0.0.1:51498_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51498.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29395f9d
   [junit4]   2> 3117096 INFO  (jetty-launcher-8192-thread-1) [n:127.0.0.1:51498_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster2-001\node1\.
   [junit4]   2> 3117118 INFO  (zkConnectionManagerCallback-8213-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3117123 INFO  (zkConnectionManagerCallback-8216-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3117125 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster1-001
   [junit4]   2> 3117125 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 3117127 INFO  (Thread-5191) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 3117127 INFO  (Thread-5191) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 3117131 ERROR (Thread-5191) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 3117228 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.ZkTestServer start zk server on port:51521
   [junit4]   2> 3117232 INFO  (zkConnectionManagerCallback-8220-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3117236 INFO  (jetty-launcher-8217-thread-1) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 1.8.0_172-b11
   [junit4]   2> 3117236 INFO  (jetty-launcher-8217-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 3117236 INFO  (jetty-launcher-8217-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 3117236 INFO  (jetty-launcher-8217-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 3117236 INFO  (jetty-launcher-8217-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@67a4d2fd{/solr,null,AVAILABLE}
   [junit4]   2> 3117237 INFO  (jetty-launcher-8217-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@139d3017{HTTP/1.1,[http/1.1]}{127.0.0.1:51525}
   [junit4]   2> 3117238 INFO  (jetty-launcher-8217-thread-1) [    ] o.e.j.s.Server Started @3117273ms
   [junit4]   2> 3117238 INFO  (jetty-launcher-8217-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51525}
   [junit4]   2> 3117238 ERROR (jetty-launcher-8217-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 3117238 INFO  (jetty-launcher-8217-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 3117238 INFO  (jetty-launcher-8217-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 3117238 INFO  (jetty-launcher-8217-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 3117238 INFO  (jetty-launcher-8217-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 3117238 INFO  (jetty-launcher-8217-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-24T22:18:52.924Z
   [junit4]   2> 3117241 INFO  (zkConnectionManagerCallback-8222-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3117242 INFO  (jetty-launcher-8217-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 3117343 INFO  (jetty-launcher-8217-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51521/solr
   [junit4]   2> 3117345 INFO  (zkConnectionManagerCallback-8226-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3117348 INFO  (zkConnectionManagerCallback-8228-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3117437 INFO  (jetty-launcher-8217-thread-1) [n:127.0.0.1:51525_solr    ] o.a.s.c.Overseer Overseer (id=null) closing
   [junit4]   2> 3117437 INFO  (jetty-launcher-8217-thread-1) [n:127.0.0.1:51525_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51525_solr
   [junit4]   2> 3117438 INFO  (jetty-launcher-8217-thread-1) [n:127.0.0.1:51525_solr    ] o.a.s.c.Overseer Overseer (id=72120976594108419-127.0.0.1:51525_solr-n_0000000000) starting
   [junit4]   2> 3117445 INFO  (zkConnectionManagerCallback-8235-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3117447 INFO  (jetty-launcher-8217-thread-1) [n:127.0.0.1:51525_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51521/solr ready
   [junit4]   2> 3117447 INFO  (jetty-launcher-8217-thread-1) [n:127.0.0.1:51525_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51525_solr
   [junit4]   2> 3117448 INFO  (OverseerStateUpdate-72120976594108419-127.0.0.1:51525_solr-n_0000000000) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3117450 INFO  (zkCallback-8234-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3117494 INFO  (jetty-launcher-8217-thread-1) [n:127.0.0.1:51525_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 3117515 INFO  (jetty-launcher-8217-thread-1) [n:127.0.0.1:51525_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51525.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29395f9d
   [junit4]   2> 3117519 INFO  (jetty-launcher-8217-thread-1) [n:127.0.0.1:51525_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51525.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29395f9d
   [junit4]   2> 3117519 INFO  (jetty-launcher-8217-thread-1) [n:127.0.0.1:51525_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51525.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29395f9d
   [junit4]   2> 3117521 INFO  (jetty-launcher-8217-thread-1) [n:127.0.0.1:51525_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster1-001\node1\.
   [junit4]   2> 3117545 INFO  (zkConnectionManagerCallback-8238-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3117549 INFO  (zkConnectionManagerCallback-8241-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3117551 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.c.CdcrBidirectionalTest cluster2 zkHost = 127.0.0.1:51494/solr
   [junit4]   2> 3117551 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.c.CdcrBidirectionalTest cluster1 zkHost = 127.0.0.1:51521/solr
   [junit4]   2> 3117554 INFO  (zkConnectionManagerCallback-8243-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3117561 INFO  (zkConnectionManagerCallback-8247-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3117562 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3117563 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51521/solr ready
   [junit4]   2> 3117564 INFO  (qtp623203464-29126) [n:127.0.0.1:51525_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=cdcr-cluster1&maxShardsPerNode=2&name=cdcr-cluster1&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 3117567 INFO  (OverseerThreadFactory-12343-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection cdcr-cluster1
   [junit4]   2> 3117674 INFO  (OverseerStateUpdate-72120976594108419-127.0.0.1:51525_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"cdcr-cluster1",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"cdcr-cluster1_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:51525/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 3117676 INFO  (OverseerStateUpdate-72120976594108419-127.0.0.1:51525_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"cdcr-cluster1",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"cdcr-cluster1_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:51525/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 3117884 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr    x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster1_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 3117884 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr    x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 3117886 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr    x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3118920 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 3118920 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 3118926 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.IndexSchema [cdcr-cluster1_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 3118926 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-cluster1_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 3118930 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 3118929 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 3118930 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster1_shard2_replica_n2' using configuration from collection cdcr-cluster1, trusted=true
   [junit4]   2> 3118930 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster1_shard1_replica_n1' using configuration from collection cdcr-cluster1, trusted=true
   [junit4]   2> 3118930 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51525.solr.core.cdcr-cluster1.shard2.replica_n2' (registry 'solr.core.cdcr-cluster1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29395f9d
   [junit4]   2> 3118930 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51525.solr.core.cdcr-cluster1.shard1.replica_n1' (registry 'solr.core.cdcr-cluster1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29395f9d
   [junit4]   2> 3118930 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 3118930 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 3118930 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore [[cdcr-cluster1_shard2_replica_n2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster1-001\node1\cdcr-cluster1_shard2_replica_n2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster1-001\node1\.\cdcr-cluster1_shard2_replica_n2\data\]
   [junit4]   2> 3118930 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-cluster1_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster1-001\node1\cdcr-cluster1_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster1-001\node1\.\cdcr-cluster1_shard1_replica_n1\data\]
   [junit4]   2> 3119137 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 3119137 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 3119137 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3119137 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3119139 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3119139 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3119139 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3119139 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3119146 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@127f5f5d[cdcr-cluster1_shard2_replica_n2] main]
   [junit4]   2> 3119146 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@797f57eb[cdcr-cluster1_shard1_replica_n1] main]
   [junit4]   2> 3119147 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster1
   [junit4]   2> 3119147 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster1
   [junit4]   2> 3119147 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster1
   [junit4]   2> 3119147 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster1
   [junit4]   2> 3119147 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster1/managed-schema
   [junit4]   2> 3119147 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster1/managed-schema
   [junit4]   2> 3119148 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 3119148 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 3119148 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3119148 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3119157 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-cluster1/cdcr/state/buffer
   [junit4]   2> 3119164 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-cluster1/cdcr/state/process
   [junit4]   2> 3119211 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604194124323356672
   [junit4]   2> 3119211 INFO  (searcherExecutor-12349-thread-1-processing-n:127.0.0.1:51525_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-cluster1_shard1_replica_n1] Registered new searcher Searcher@797f57eb[cdcr-cluster1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3119211 INFO  (searcherExecutor-12348-thread-1-processing-n:127.0.0.1:51525_solr x:cdcr-cluster1_shard2_replica_n2 c:cdcr-cluster1 s:shard2 r:core_node4) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore [cdcr-cluster1_shard2_replica_n2] Registered new searcher Searcher@127f5f5d[cdcr-cluster1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3119211 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604194124323356672
   [junit4]   2> 3119216 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 3119217 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard2 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 3119218 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3119218 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3119218 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51525/solr/cdcr-cluster1_shard1_replica_n1/
   [junit4]   2> 3119219 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3119219 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:51525/solr/cdcr-cluster1_shard1_replica_n1/ has no replicas
   [junit4]   2> 3119219 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 3119219 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3119219 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3119219 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51525/solr/cdcr-cluster1_shard2_replica_n2/
   [junit4]   2> 3119220 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3119220 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:51525/solr/cdcr-cluster1_shard2_replica_n2/ has no replicas
   [junit4]   2> 3119220 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 3119222 INFO  (zkCallback-8227-thread-1) [    ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster1:shard1
   [junit4]   2> 3119224 INFO  (zkCallback-8227-thread-1) [    ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster1:shard2
   [junit4]   2> 3119225 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51525/solr/cdcr-cluster1_shard1_replica_n1/ shard1
   [junit4]   2> 3119229 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51525/solr/cdcr-cluster1_shard2_replica_n2/ shard2
   [junit4]   2> 3119380 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3119381 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3119384 INFO  (qtp623203464-29122) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1502
   [junit4]   2> 3119385 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster1_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1503
   [junit4]   2> 3119388 INFO  (qtp623203464-29126) [n:127.0.0.1:51525_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 3119486 INFO  (zkCallback-8227-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/state.json] for collection [cdcr-cluster1] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3119569 INFO  (OverseerCollectionConfigSetProcessor-72120976594108419-127.0.0.1:51525_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 3120388 INFO  (qtp623203464-29126) [n:127.0.0.1:51525_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=cdcr-cluster1&maxShardsPerNode=2&name=cdcr-cluster1&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2824
   [junit4]   2> 3120392 INFO  (zkConnectionManagerCallback-8251-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3120398 INFO  (zkConnectionManagerCallback-8255-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3120400 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3120400 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51494/solr ready
   [junit4]   2> 3120401 INFO  (qtp2005352843-29072) [n:127.0.0.1:51498_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=cdcr-cluster2&maxShardsPerNode=2&name=cdcr-cluster2&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 3120404 INFO  (OverseerThreadFactory-12330-thread-1) [    ] o.a.s.c.a.c.CreateCollectionCmd Create collection cdcr-cluster2
   [junit4]   2> 3120511 INFO  (OverseerStateUpdate-72120976566452227-127.0.0.1:51498_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"cdcr-cluster2",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"cdcr-cluster2_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:51498/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 3120513 INFO  (OverseerStateUpdate-72120976566452227-127.0.0.1:51498_solr-n_0000000000) [    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"cdcr-cluster2",
   [junit4]   2>   "shard":"shard2",
   [junit4]   2>   "core":"cdcr-cluster2_shard2_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"http://127.0.0.1:51498/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 3120719 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr    x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster2_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 3120719 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr    x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster2_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
   [junit4]   2> 3120720 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr    x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 3121751 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 3121751 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 3121756 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-cluster2_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 3121756 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.IndexSchema [cdcr-cluster2_shard2_replica_n2] Schema name=minimal
   [junit4]   2> 3121761 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 3121761 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 3121761 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster2_shard2_replica_n2' using configuration from collection cdcr-cluster2, trusted=true
   [junit4]   2> 3121761 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster2_shard1_replica_n1' using configuration from collection cdcr-cluster2, trusted=true
   [junit4]   2> 3121761 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51498.solr.core.cdcr-cluster2.shard1.replica_n1' (registry 'solr.core.cdcr-cluster2.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29395f9d
   [junit4]   2> 3121761 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51498.solr.core.cdcr-cluster2.shard2.replica_n2' (registry 'solr.core.cdcr-cluster2.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29395f9d
   [junit4]   2> 3121761 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 3121763 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 3121763 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-cluster2_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster2-001\node1\cdcr-cluster2_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster2-001\node1\.\cdcr-cluster2_shard1_replica_n1\data\]
   [junit4]   2> 3121763 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore [[cdcr-cluster2_shard2_replica_n2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster2-001\node1\cdcr-cluster2_shard2_replica_n2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.cdcr.CdcrBidirectionalTest_E3B508AB83D18620-001\cdcr-cluster2-001\node1\.\cdcr-cluster2_shard2_replica_n2\data\]
   [junit4]   2> 3121897 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 3121897 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
   [junit4]   2> 3121897 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3121897 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 3121899 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3121899 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 3121900 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3121900 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 3121901 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@686c39eb[cdcr-cluster2_shard1_replica_n1] main]
   [junit4]   2> 3121901 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@2b040dd9[cdcr-cluster2_shard2_replica_n2] main]
   [junit4]   2> 3121903 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster2
   [junit4]   2> 3121903 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster2
   [junit4]   2> 3121903 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster2
   [junit4]   2> 3121903 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster2/managed-schema
   [junit4]   2> 3121903 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster2
   [junit4]   2> 3121903 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster2/managed-schema
   [junit4]   2> 3121904 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 3121904 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
   [junit4]   2> 3121904 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3121904 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 3121906 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-cluster2/cdcr/state/buffer
   [junit4]   2> 3121907 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-cluster2/cdcr/state/process
   [junit4]   2> 3121911 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604194127154511872
   [junit4]   2> 3121911 INFO  (searcherExecutor-12361-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-cluster2_shard1_replica_n1] Registered new searcher Searcher@686c39eb[cdcr-cluster2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3121911 INFO  (searcherExecutor-12360-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore [cdcr-cluster2_shard2_replica_n2] Registered new searcher Searcher@2b040dd9[cdcr-cluster2_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3121911 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604194127154511872
   [junit4]   2> 3121917 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster2/terms/shard2 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 3121918 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster2/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 3121920 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3121920 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3121920 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51498/solr/cdcr-cluster2_shard2_replica_n2/
   [junit4]   2> 3121920 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3121921 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 3121921 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:51498/solr/cdcr-cluster2_shard2_replica_n2/ has no replicas
   [junit4]   2> 3121921 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 3121921 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 3121921 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51498/solr/cdcr-cluster2_shard1_replica_n1/
   [junit4]   2> 3121921 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
   [junit4]   2> 3121922 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:51498/solr/cdcr-cluster2_shard1_replica_n1/ has no replicas
   [junit4]   2> 3121922 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 3121923 INFO  (zkCallback-8202-thread-1) [    ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster2:shard2
   [junit4]   2> 3121925 INFO  (zkCallback-8202-thread-2) [    ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster2:shard1
   [junit4]   2> 3121929 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51498/solr/cdcr-cluster2_shard2_replica_n2/ shard2
   [junit4]   2> 3121929 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51498/solr/cdcr-cluster2_shard1_replica_n1/ shard1
   [junit4]   2> 3122033 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3122037 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster2_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1318
   [junit4]   2> 3122083 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 3122088 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster2_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1370
   [junit4]   2> 3122090 INFO  (qtp2005352843-29072) [n:127.0.0.1:51498_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 3122186 INFO  (zkCallback-8202-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster2/state.json] for collection [cdcr-cluster2] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 3122406 INFO  (OverseerCollectionConfigSetProcessor-72120976566452227-127.0.0.1:51498_solr-n_0000000000) [    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 3123090 INFO  (qtp2005352843-29072) [n:127.0.0.1:51498_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=cdcr-cluster2&maxShardsPerNode=2&name=cdcr-cluster2&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2688
   [junit4]   2> 3123101 INFO  (zkConnectionManagerCallback-8262-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3123102 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3123103 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51494/solr ready
   [junit4]   2> 3123110 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
   [junit4]   2> 3123110 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
   [junit4]   2> 3123110 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1]  webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 3123110 INFO  (qtp2005352843-29068) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2]  webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 3123110 INFO  (qtp2005352843-29075) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2]  webapp=/solr path=/cdcr params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 3123110 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard1
   [junit4]   2> 3123110 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2, shard: shard1
   [junit4]   2> 3123115 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
   [junit4]   2> 3123116 INFO  (zkCallback-8227-thread-1) [    ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/cdcr/state/process @ cdcr-cluster1:shard1
   [junit4]   2> 3123116 INFO  (zkCallback-8227-thread-2) [    ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/cdcr/state/process @ cdcr-cluster1:shard2
   [junit4]   2> 3123116 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1]  webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-cluster1:4&action=start&wt=javabin&version=2} status=0 QTime=20
   [junit4]   2> 3123116 INFO  (zkCallback-8227-thread-2) [    ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-cluster1:shard2
   [junit4]   2> 3123116 INFO  (zkCallback-8227-thread-1) [    ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-cluster1:shard1
   [junit4]   2> 3123116 INFO  (cdcr-bootstrap-status-8258-thread-1-processing-n:127.0.0.1:51525_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2 shard: shard1 leader: http://127.0.0.1:51498/solr/cdcr-cluster2_shard1_replica_n1/
   [junit4]   2> 3123118 INFO  (qtp2005352843-29070) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1]  webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=http://127.0.0.1:51525/solr/cdcr-cluster1_shard1_replica_n1/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 3123122 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1]  webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 3123122 INFO  (cdcr-bootstrap-status-8258-thread-1-processing-n:127.0.0.1:51525_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager CDCR bootstrap running for 1 seconds, sleeping for 2000 ms
   [junit4]   2> 3123122 INFO  (zkConnectionManagerCallback-8267-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 3123124 INFO  (zkCallback-8227-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 3123124 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 3123124 INFO  (zkCallback-8227-thread-2) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51494/solr ready
   [junit4]   2> 3123127 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604194128429580288,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 3123128 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 3123129 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
   [junit4]   2> 3123129 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2]  webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 3123129 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 3123129 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1]  webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2} status=0 QTime=3
   [junit4]   2> 3123131 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 3123131 INFO  (qtp2005352843-29074) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
   [junit4]   2> 3123131 INFO  (qtp2005352843-29074) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1]  webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 3123131 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 3123131 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 3123131 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 3123131 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 3123131 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
   [junit4]   2> 3123132 INFO  (qtp2005352843-29070) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2]  webapp=/solr path=/cdcr params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=6
   [junit4]   2> 3123132 INFO  (zkCallback-8227-thread-2) [    ] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard2
   [junit4]   2> 3123132 INFO  (zkCallback-8227-thread-2) [    ] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2, shard: shard2
   [junit4]   2> 3123132 INFO  (zkCallback-8227-thread-2) [    ] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
   [junit4]   2> 3123132 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@29cb6e75[cdcr-cluster2_shard1_replica_n1] main]
   [junit4]   2> 3123133 INFO  (cdcr-bootstrap-status-8263-thread-1) [    ] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2 shard: shard2 leader: http://127.0.0.1:51498/solr/cdcr-cluster2_shard2_replica_n2/
   [junit4]   2> 3123134 INFO  (searcherExecutor-12361-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-cluster2_shard1_replica_n1] Registered new searcher Searcher@29cb6e75[cdcr-cluster2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3123134 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler No replay needed.
   [junit4]   2> 3123134 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2]  webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=http://127.0.0.1:51525/solr/cdcr-cluster1_shard2_replica_n2/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 3123134 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
   [junit4]   2> 3123134 INFO  (qtp2005352843-29074) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2]  webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 3123135 INFO  (cdcr-bootstrap-status-8263-thread-1) [    ] o.a.s.h.CdcrReplicatorManager CDCR bootstrap running for 1 seconds, sleeping for 2000 ms
   [junit4]   2> 3123136 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604194128439017472,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 3123137 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
   [junit4]   2> 3123138 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 3123138 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2]  webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 3123139 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 3123140 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 3123140 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 3123140 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 3123140 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 3123140 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
   [junit4]   2> 3123140 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@43046d0c[cdcr-cluster2_shard2_replica_n2] main]
   [junit4]   2> 3123141 INFO  (searcherExecutor-12360-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore [cdcr-cluster2_shard2_replica_n2] Registered new searcher Searcher@43046d0c[cdcr-cluster2_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 3123141 INFO  (recoveryExecutor-8199-thread-1-processing-n:127.0.0.1:51498_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrRequestHandler No replay needed.
   [junit4]   2> 3125117 INFO  (TEST-CdcrBidirectionalTest.testBiDir-seed#[E3B508AB83D18620]) [    ] o.a.s.c.c.CdcrBidirectionalTest Adding 10 docs with commit=true, numDocs=100
   [junit4]   2> 3125125 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard1 to Terms{values={core_node3=1}, version=1}
   [junit4]   2> 3125126 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1]  webapp=/solr path=/update params={_stateVer_=cdcr-cluster1:4&wt=javabin&version=2} status=0 QTime=5
   [junit4]   2> 3125126 INFO  (qtp2005352843-29074) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1]  webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 3125126 INFO  (cdcr-bootstrap-status-8258-thread-1-processing-n:127.0.0.1:51525_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager CDCR bootstrap successful in 3 seconds
   [junit4]   2> 3125126 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard2 to Terms{values={core_node4=1}, version=1}
   [junit4]   2> 3125127 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2]  webapp=/solr path=/update params={_stateVer_=cdcr-cluster1:4&wt=javabin&version=2} status=0 QTime=7
   [junit4]   2> 3125132 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
   [junit4]   2> 3125132 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2]  webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 3125134 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
   [junit4]   2> 3125134 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1]  webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 3125135 INFO  (qtp2005352843-29074) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1]  webapp=/solr path=/cdcr params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=8
   [junit4]   2> 3125135 INFO  (cdcr-bootstrap-status-8258-thread-1-processing-n:127.0.0.1:51525_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard1
   [junit4]   2> 3125136 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604194130536169472,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 3125136 INFO  (qtp623203464-29129) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@7a60ee22 commitCommandVersion:1604194130536169472
   [junit4]   2> 3125137 INFO  (cdcr-bootstrap-status-8258-thread-1-processing-n:127.0.0.1:51525_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Bootstrap successful, giving the go-ahead to replicator
   [junit4]   2> 3125139 INFO  (qtp2005352843-29071) [n:127.0.0.1:51498_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2]  webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 3125139 INFO  (cdcr-bootstrap-status-8263-thread-1) [    ] o.a.s.h.CdcrReplicatorManager CDCR bootstrap successful in 3 seconds
   [junit4]   2> 3125145 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604194130545606656,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 3125145 INFO  (qtp623203464-29127) [n:127.0.0.1:51525_solr c:cdcr-cluster1 s:shard

[...truncated too long message...]

ails.

compile-test:

BUILD SUCCESSFUL
Total time: 24 seconds
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestTimeSource" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC" -Dtests.seed=51F3195FCA7E1898 -Dtests.slow=true -Dtests.locale=sq-AL -Dtests.timezone=Europe/Saratov -Dtests.asserts=true -Dtests.file.encoding=UTF-8
Buildfile: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\solrj\build.xml

-clover.disable:

ivy-configure:
[ivy:configure] :: Apache Ivy 2.4.0 - 20141213170938 :: http://ant.apache.org/ivy/ ::
[ivy:configure] :: loading settings :: file = C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\top-level-ivy-settings.xml

install-junit4-taskdef:

ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.

-ivy-fail-disallowed-ivy-version:

ivy-fail:

resolve-groovy:
[ivy:cachepath] :: resolving dependencies :: org.codehaus.groovy#groovy-all-caller;working
[ivy:cachepath] 	confs: [default]
[ivy:cachepath] 	found org.codehaus.groovy#groovy-all;2.4.15 in public
[ivy:cachepath] :: resolution report :: resolve 11ms :: artifacts dl 0ms
	---------------------------------------------------------------------
	|                  |            modules            ||   artifacts   |
	|       conf       | number| search|dwnlded|evicted|| number|dwnlded|
	---------------------------------------------------------------------
	|      default     |   1   |   0   |   0   |   0   ||   1   |   0   |
	---------------------------------------------------------------------

-init-totals:

-test:
    [mkdir] Created dir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-solrj\test
[junit4:pickseed] Seed property 'tests.seed' already defined: 51F3195FCA7E1898
    [mkdir] Created dir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-solrj\test\temp
   [junit4] <JUnit4> says ciao! Master seed: 51F3195FCA7E1898
   [junit4] Your default console's encoding may not display certain unicode glyphs: windows-1252
   [junit4] Executing 5 suites with 2 JVMs.
   [junit4] 
   [junit4] Started J1 PID(29608@serv1-vm1).
   [junit4] Started J0 PID(13100@serv1-vm1).
   [junit4] Suite: org.apache.solr.common.util.TestTimeSource
   [junit4]   2> SLF4J: Class path contains multiple SLF4J bindings.
   [junit4]   2> SLF4J: Found binding in [jar:file:/C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/solrj/test-lib/log4j-slf4j-impl-2.11.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   [junit4]   2> SLF4J: Found binding in [jar:file:/C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/core/lib/log4j-slf4j-impl-2.11.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
   [junit4]   2> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
   [junit4]   2> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
   [junit4]   2> 13632 INFO  (SUITE-TestTimeSource-seed#[51F3195FCA7E1898]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-solrj\test\J0\temp\solr.common.util.TestTimeSource_51F3195FCA7E1898-001\init-core-data-001
   [junit4]   2> 14727 INFO  (SUITE-TestTimeSource-seed#[51F3195FCA7E1898]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 15088 INFO  (SUITE-TestTimeSource-seed#[51F3195FCA7E1898]-worker) [    ] o.e.j.u.log Logging initialized @15154ms to org.eclipse.jetty.util.log.Slf4jLog
   [junit4]   2> 15154 INFO  (SUITE-TestTimeSource-seed#[51F3195FCA7E1898]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
   [junit4]   2> 15219 INFO  (TEST-TestTimeSource.testEpochTime-seed#[51F3195FCA7E1898]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testEpochTime
   [junit4]   2> 25290 INFO  (TEST-TestTimeSource.testEpochTime-seed#[51F3195FCA7E1898]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testEpochTime
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestTimeSource -Dtests.method=testEpochTime -Dtests.seed=51F3195FCA7E1898 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=sq-AL -Dtests.timezone=Europe/Saratov -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 10.2s J0 | TestTimeSource.testEpochTime <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: SimTimeSource:50.0 time diff=53185000
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([51F3195FCA7E1898:699F6A7A5EAEBADE]:0)
   [junit4]    > 	at org.apache.solr.common.util.TestTimeSource.doTestEpochTime(TestTimeSource.java:52)
   [junit4]    > 	at org.apache.solr.common.util.TestTimeSource.testEpochTime(TestTimeSource.java:32)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-solrj\test\J0\temp\solr.common.util.TestTimeSource_51F3195FCA7E1898-001
   [junit4]   2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=30239, maxDocsPerChunk=301, blockSize=234), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=30239, blockSize=234)), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@64e550c2), locale=sq-AL, timezone=Europe/Saratov
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=3,threads=1,free=79518848,total=93323264
   [junit4]   2> NOTE: All tests run in this JVM: [TestTimeSource]
   [junit4] Completed [1/5 (1!)] on J0 in 23.47s, 1 test, 1 failure <<< FAILURES!
   [junit4] 
   [junit4] Suite: org.apache.solr.common.util.TestTimeSource
   [junit4] OK      10.4s J1 | TestTimeSource.testEpochTime
   [junit4] Completed [2/5 (1!)] on J1 in 23.62s, 1 test
   [junit4] 
   [junit4] Duplicate suite name used with XML reports: org.apache.solr.common.util.TestTimeSource. This may confuse tools that process XML reports. Set 'ignoreDuplicateSuites' to true to skip this message.
   [junit4] Suite: org.apache.solr.common.util.TestTimeSource
   [junit4] OK      10.2s J0 | TestTimeSource.testEpochTime
   [junit4] Completed [3/5 (1!)] on J0 in 10.32s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.common.util.TestTimeSource
   [junit4] OK      10.1s J1 | TestTimeSource.testEpochTime
   [junit4] Completed [4/5 (1!)] on J1 in 10.34s, 1 test
   [junit4] 
   [junit4] Suite: org.apache.solr.common.util.TestTimeSource
   [junit4] OK      10.1s J0 | TestTimeSource.testEpochTime
   [junit4] Completed [5/5 (1!)] on J0 in 10.19s, 1 test
   [junit4] 
   [junit4] 
   [junit4] Tests with failures [seed: 51F3195FCA7E1898]:
   [junit4]   - org.apache.solr.common.util.TestTimeSource.testEpochTime
   [junit4] 
   [junit4] 
   [junit4] JVM J0:     1.42 ..    46.84 =    45.42s
   [junit4] JVM J1:     1.36 ..    37.23 =    35.87s
   [junit4] Execution time total: 46 seconds
   [junit4] Tests summary: 5 suites, 5 tests, 1 failure

BUILD FAILED
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\common-build.xml:1568: The following error occurred while executing this line:
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\lucene\common-build.xml:1092: There were test failures: 5 suites, 5 tests, 1 failure [seed: 51F3195FCA7E1898]

Total time: 49 seconds

[repro] Setting last failure code to 1

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest
[repro]   0/5 failed: org.apache.solr.schema.TestUseDocValuesAsStored2
[repro]   1/5 failed: org.apache.solr.common.util.TestTimeSource
[repro] Exiting with code 1

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows>IF EXIST lucene\build move /y lucene\build lucene\build.repro 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows>IF EXIST solr\build move /y solr\build solr\build.repro 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows>IF EXIST lucene\build.orig move /y lucene\build.orig lucene\build 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows>IF EXIST solr\build.orig move /y solr\build.orig solr\build 
        1 dir(s) moved.

C:\Users\jenkins\workspace\Lucene-Solr-master-Windows>exit 0 
Archiving artifacts
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=3b9d3a760a432b97aad2c08b2f778fa2344eb14a, workspace=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows
[WARNINGS] Computing warning deltas based on reference build #7377
Recording test results
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2
Setting ANT_1_8_2_HOME=C:\Users\jenkins\tools\hudson.tasks.Ant_AntInstallation\ANT_1.8.2

[JENKINS] Lucene-Solr-master-Windows (64bit/jdk-10) - Build # 7379 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7379/
Java: 64bit/jdk-10 -XX:+UseCompressedOops -XX:+UseParallelGC

2 tests failed.
FAILED:  org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup

Error Message:
Problem reading index. (resource=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\collection1\data\snapshot.ljnwxezizdkto\_0.fnm)

Stack Trace:
org.apache.lucene.index.CorruptIndexException: Problem reading index. (resource=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\collection1\data\snapshot.ljnwxezizdkto\_0.fnm)
	at __randomizedtesting.SeedInfo.seed([78CC2F61B83F28D7:39470F049F81DB98]:0)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:143)
	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:82)
	at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:65)
	at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:57)
	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680)
	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:80)
	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
	at org.apache.solr.handler.TestReplicationHandlerBackup.verify(TestReplicationHandlerBackup.java:146)
	at org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup(TestReplicationHandlerBackup.java:210)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: java.nio.file.NoSuchFileException: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\collection1\data\snapshot.ljnwxezizdkto\_0.fnm
	at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
	at java.base/sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:231)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:369)
	at java.base/java.nio.file.Files.newByteChannel(Files.java:415)
	at org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:77)
	at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:121)
	at org.apache.lucene.codecs.lucene60.Lucene60FieldInfosFormat.read(Lucene60FieldInfosFormat.java:113)
	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:108)
	... 46 more


FAILED:  org.apache.solr.common.util.TestTimeSource.testEpochTime

Error Message:
SimTimeSource:50.0 time diff=14545000

Stack Trace:
java.lang.AssertionError: SimTimeSource:50.0 time diff=14545000
	at __randomizedtesting.SeedInfo.seed([16930F028652EAED:2EFF7C27128248AB]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.common.util.TestTimeSource.doTestEpochTime(TestTimeSource.java:52)
	at org.apache.solr.common.util.TestTimeSource.testEpochTime(TestTimeSource.java:32)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 14178 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandlerBackup
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\init-core-data-001
   [junit4]   2> 1527811 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10+46
   [junit4]   2> 1527812 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1527812 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1527812 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1527812 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2cd42e2f{/solr,null,AVAILABLE}
   [junit4]   2> 1527814 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@209d14de{HTTP/1.1,[http/1.1]}{127.0.0.1:58466}
   [junit4]   2> 1527814 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.Server Started @1527847ms
   [junit4]   2> 1527814 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\collection1\data, hostPort=58466}
   [junit4]   2> 1527814 ERROR (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1527814 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1527814 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1527814 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 1527814 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1527814 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-25T10:34:30.661834300Z
   [junit4]   2> 1527814 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\solr.xml
   [junit4]   2> 1527816 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1527816 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1527817 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@45d3f7f5, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1528116 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1528116 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1528140 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45d3f7f5
   [junit4]   2> 1528149 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45d3f7f5
   [junit4]   2> 1528149 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45d3f7f5
   [junit4]   2> 1528154 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\.
   [junit4]   2> 1528154 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 1528190 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1528194 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1528195 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
   [junit4]   2> 1528196 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\.\collection1, trusted=true
   [junit4]   2> 1528196 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45d3f7f5
   [junit4]   2> 1528196 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1528196 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\collection1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\.\collection1\data\]
   [junit4]   2> 1528197 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=12.47265625, floorSegmentMB=0.841796875, forceMergeDeletesPctAllowed=20.745456692198847, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8243489172847611, reclaimDeletesWeight=0.19988820802168972
   [junit4]   2> 1528285 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1528285 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1528286 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=46.3720703125, floorSegmentMB=0.431640625, forceMergeDeletesPctAllowed=10.961003557652731, segmentsPerTier=34.0, maxCFSSegmentSizeMB=0.8505859375, noCFSRatio=0.1675172673936574, reclaimDeletesWeight=2.370816352645129
   [junit4]   2> 1528287 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1c58dc86[collection1] main]
   [junit4]   2> 1528288 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\collection1\conf
   [junit4]   2> 1528289 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema-replication2.xml:schema.xml
   [junit4]   2> 1528289 INFO  (coreLoadExecutor-6523-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1528290 INFO  (searcherExecutor-6524-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1c58dc86[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1528795 INFO  (qtp7921683-14540) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1528795 INFO  (qtp7921683-14540) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
   [junit4]   2> 1528795 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.h.BackupRestoreUtils Indexing 24 test docs
   [junit4]   2> 1528796 INFO  (qtp7921683-14540) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ... (24 adds)]} 0 0
   [junit4]   2> 1528797 INFO  (qtp7921683-14540) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1528797 INFO  (qtp7921683-14540) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2ebd0530 commitCommandVersion:0
   [junit4]   2> 1528801 INFO  (Thread-2899) [    ] o.a.s.h.SnapShooter Creating backup snapshot <not named> at file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1528801 INFO  (qtp7921683-14540) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5bdc20fa[collection1] main]
   [junit4]   2> 1528803 INFO  (searcherExecutor-6524-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5bdc20fa[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C24)))}
   [junit4]   2> 1528803 INFO  (qtp7921683-14540) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1528804 INFO  (qtp7921683-14540) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 6
   [junit4]   2> 1528806 INFO  (qtp7921683-14536) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={wt=xml&command=details} status=0 QTime=0
   [junit4]   2> 1528812 INFO  (Thread-2899) [    ] o.a.s.h.SnapShooter Done creating backup snapshot: <not named> at file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1529806 INFO  (qtp7921683-14539) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={wt=xml&command=details} status=0 QTime=0
   [junit4]   2> 1530809 INFO  (qtp7921683-14542) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={name=neofkzewonjgttzmp&wt=xml&command=backup} status=0 QTime=0
   [junit4]   2> 1530811 INFO  (Thread-2900) [    ] o.a.s.h.SnapShooter Creating backup snapshot neofkzewonjgttzmp at file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1530815 INFO  (qtp7921683-14539) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={wt=xml&command=details} status=0 QTime=3
   [junit4]   2> 1530963 INFO  (Thread-2900) [    ] o.a.s.h.SnapShooter Done creating backup snapshot: neofkzewonjgttzmp at file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1531819 INFO  (qtp7921683-14542) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={name=ddzm&wt=xml&command=backup} status=0 QTime=0
   [junit4]   2> 1531822 INFO  (Thread-2901) [    ] o.a.s.h.SnapShooter Creating backup snapshot ddzm at file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1531824 INFO  (qtp7921683-14542) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={wt=xml&command=details} status=0 QTime=3
   [junit4]   2> 1531831 INFO  (Thread-2901) [    ] o.a.s.h.SnapShooter Done creating backup snapshot: ddzm at file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1532831 INFO  (qtp7921683-14539) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={name=ljnwxezizdkto&wt=xml&command=backup} status=0 QTime=0
   [junit4]   2> 1532831 INFO  (Thread-2902) [    ] o.a.s.h.SnapShooter Creating backup snapshot ljnwxezizdkto at file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1532831 INFO  (qtp7921683-14542) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={wt=xml&command=details} status=0 QTime=0
   [junit4]   2> 1533933 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@209d14de{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1533934 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=461040576
   [junit4]   2> 1533934 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1533934 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4cfd6e19: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@43a2189b
   [junit4]   2> 1533943 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1533943 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@799377fc: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1808be7e
   [junit4]   2> 1533949 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1533949 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5c3ee6bb: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@306b28ac
   [junit4]   2> 1533955 INFO  (coreCloseExecutor-6528-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@36f00bae
   [junit4]   2> 1533955 INFO  (coreCloseExecutor-6528-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=36f00bae
   [junit4]   2> 1533955 INFO  (coreCloseExecutor-6528-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@790ae93b: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@7fe1639a
   [junit4]   2> 1533964 INFO  (Thread-2902) [    ] o.a.s.h.SnapShooter Done creating backup snapshot: ljnwxezizdkto at file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001/solr-instance-001/./collection1/data/
   [junit4]   2> 1533966 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2cd42e2f{/solr,null,UNAVAILABLE}
   [junit4]   2> 1533966 INFO  (TEST-TestReplicationHandlerBackup.doTestBackup-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandlerBackup -Dtests.method=doTestBackup -Dtests.seed=78CC2F61B83F28D7 -Dtests.slow=true -Dtests.locale=eu -Dtests.timezone=America/North_Dakota/Center -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] ERROR   6.17s J1 | TestReplicationHandlerBackup.doTestBackup <<<
   [junit4]    > Throwable #1: org.apache.lucene.index.CorruptIndexException: Problem reading index. (resource=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\collection1\data\snapshot.ljnwxezizdkto\_0.fnm)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([78CC2F61B83F28D7:39470F049F81DB98]:0)
   [junit4]    > 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:143)
   [junit4]    > 	at org.apache.lucene.index.SegmentReader.<init>(SegmentReader.java:82)
   [junit4]    > 	at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:65)
   [junit4]    > 	at org.apache.lucene.index.StandardDirectoryReader$1.doBody(StandardDirectoryReader.java:57)
   [junit4]    > 	at org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(SegmentInfos.java:680)
   [junit4]    > 	at org.apache.lucene.index.StandardDirectoryReader.open(StandardDirectoryReader.java:80)
   [junit4]    > 	at org.apache.lucene.index.DirectoryReader.open(DirectoryReader.java:63)
   [junit4]    > 	at org.apache.solr.handler.TestReplicationHandlerBackup.verify(TestReplicationHandlerBackup.java:146)
   [junit4]    > 	at org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup(TestReplicationHandlerBackup.java:210)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]    > Caused by: java.nio.file.NoSuchFileException: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-001\collection1\data\snapshot.ljnwxezizdkto\_0.fnm
   [junit4]    > 	at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:85)
   [junit4]    > 	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
   [junit4]    > 	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
   [junit4]    > 	at java.base/sun.nio.fs.WindowsFileSystemProvider.newByteChannel(WindowsFileSystemProvider.java:231)
   [junit4]    > 	at java.base/java.nio.file.Files.newByteChannel(Files.java:369)
   [junit4]    > 	at java.base/java.nio.file.Files.newByteChannel(Files.java:415)
   [junit4]    > 	at org.apache.lucene.store.SimpleFSDirectory.openInput(SimpleFSDirectory.java:77)
   [junit4]    > 	at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:121)
   [junit4]    > 	at org.apache.lucene.codecs.lucene60.Lucene60FieldInfosFormat.read(Lucene60FieldInfosFormat.java:113)
   [junit4]    > 	at org.apache.lucene.index.SegmentCoreReaders.<init>(SegmentCoreReaders.java:108)
   [junit4]    > 	... 46 more
   [junit4]   2> 1533978 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.Server jetty-9.4.10.v20180503; built: 2018-05-03T15:56:21.710Z; git: daa59876e6f384329b122929e70a80934569428c; jvm 10+46
   [junit4]   2> 1533978 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1533978 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1533978 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.session node0 Scavenging every 600000ms
   [junit4]   2> 1533979 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@754e4193{/solr,null,AVAILABLE}
   [junit4]   2> 1533980 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3401aa2f{HTTP/1.1,[http/1.1]}{127.0.0.1:58500}
   [junit4]   2> 1533981 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.Server Started @1534013ms
   [junit4]   2> 1533981 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, solr.data.dir=C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-002\collection1\data, hostPort=58500}
   [junit4]   2> 1533981 ERROR (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1533981 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1533981 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 1533981 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in standalone mode on port null
   [junit4]   2> 1533981 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1533981 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-06-25T10:34:36.828108100Z
   [junit4]   2> 1533981 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.SolrXmlConfig Loading container configuration from C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-002\solr.xml
   [junit4]   2> 1533983 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
   [junit4]   2> 1533984 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
   [junit4]   2> 1533984 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@45d3f7f5, but no JMX reporters were configured - adding default JMX reporter.
   [junit4]   2> 1534452 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
   [junit4]   2> 1534452 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1534468 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45d3f7f5
   [junit4]   2> 1534475 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45d3f7f5
   [junit4]   2> 1534476 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45d3f7f5
   [junit4]   2> 1534478 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-002\.
   [junit4]   2> 1534478 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
   [junit4]   2> 1534486 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 1534496 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
   [junit4]   2> 1534497 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
   [junit4]   2> 1534497 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-002\.\collection1, trusted=true
   [junit4]   2> 1534497 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@45d3f7f5
   [junit4]   2> 1534497 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
   [junit4]   2> 1534497 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-002\collection1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-002\.\collection1\data\]
   [junit4]   2> 1534499 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=30, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=12.47265625, floorSegmentMB=0.841796875, forceMergeDeletesPctAllowed=20.745456692198847, segmentsPerTier=24.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8243489172847611, reclaimDeletesWeight=0.19988820802168972
   [junit4]   2> 1534568 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1534568 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1534569 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=20, maxMergeAtOnceExplicit=35, maxMergedSegmentMB=46.3720703125, floorSegmentMB=0.431640625, forceMergeDeletesPctAllowed=10.961003557652731, segmentsPerTier=34.0, maxCFSSegmentSizeMB=0.8505859375, noCFSRatio=0.1675172673936574, reclaimDeletesWeight=2.370816352645129
   [junit4]   2> 1534569 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4326eaca[collection1] main]
   [junit4]   2> 1534569 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001\solr-instance-002\collection1\conf
   [junit4]   2> 1534570 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema-replication2.xml:schema.xml
   [junit4]   2> 1534570 INFO  (coreLoadExecutor-6538-thread-1) [    x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1534570 INFO  (searcherExecutor-6539-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4326eaca[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1534572 INFO  (qtp1197829789-14569) [    x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 1534573 INFO  (qtp1197829789-14569) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
   [junit4]   2> 1534573 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.h.BackupRestoreUtils Indexing 46 test docs
   [junit4]   2> 1534575 INFO  (qtp1197829789-14565) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0, 1, 2, 3, 4, 5, 6, 7, 8, 9, ... (46 adds)]} 0 1
   [junit4]   2> 1534576 INFO  (qtp1197829789-14568) [    x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1534576 INFO  (qtp1197829789-14568) [    x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2a6157ab commitCommandVersion:0
   [junit4]   2> 1534578 INFO  (Thread-2913) [    ] o.a.s.h.SnapShooter Creating backup snapshot <not named> at file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001/solr-instance-002/./collection1/data/
   [junit4]   2> 1534578 INFO  (qtp1197829789-14568) [    x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2fe8b195[collection1] main]
   [junit4]   2> 1534578 INFO  (qtp1197829789-14568) [    x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1534579 INFO  (searcherExecutor-6539-thread-1-processing-x:collection1) [    x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@2fe8b195[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C46)))}
   [junit4]   2> 1534579 INFO  (qtp1197829789-14568) [    x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1]  webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 4
   [junit4]   2> 1534583 INFO  (qtp1197829789-14570) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={wt=xml&command=details} status=0 QTime=3
   [junit4]   2> 1534588 INFO  (Thread-2913) [    ] o.a.s.h.SnapShooter Done creating backup snapshot: <not named> at file:///C:/Users/jenkins/workspace/Lucene-Solr-master-Windows/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001/solr-instance-002/./collection1/data/
   [junit4]   2> 1535583 INFO  (qtp1197829789-14568) [    x:collection1] o.a.s.c.S.Request [collection1]  webapp=/solr path=/replication params={wt=xml&command=details} status=0 QTime=0
   [junit4]   2> 1536588 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@3401aa2f{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1536588 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1894985474
   [junit4]   2> 1536589 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1536589 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4719d7ed: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@254f475e
   [junit4]   2> 1536593 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1536593 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2c8a3c86: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1808be7e
   [junit4]   2> 1536595 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1536596 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3ed2c630: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@306b28ac
   [junit4]   2> 1536596 INFO  (coreCloseExecutor-6543-thread-1) [    x:collection1] o.a.s.c.SolrCore [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@2cc1e6fd
   [junit4]   2> 1536596 INFO  (coreCloseExecutor-6543-thread-1) [    x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=2cc1e6fd
   [junit4]   2> 1536596 INFO  (coreCloseExecutor-6543-thread-1) [    x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@633b172e: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@89e4bd7
   [junit4]   2> 1536603 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@754e4193{/solr,null,UNAVAILABLE}
   [junit4]   2> 1536603 INFO  (TEST-TestReplicationHandlerBackup.testBackupOnCommit-seed#[78CC2F61B83F28D7]) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J1\temp\solr.handler.TestReplicationHandlerBackup_78CC2F61B83F28D7-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {name=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=LuceneVarGapDocFreqInterval)}, docValues:{}, maxPointsInLeafNode=1756, maxMBSortInHeap=7.715985297530205, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@181d5d9f), locale=eu, timezone=America/North_Dakota/Center
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 10 (64-bit)/cpus=3,threads=1,free=207834408,total=424148992
   [junit4]   2> NOTE: All tests run in this JVM: [TestSubQueryTransformer, SimplePostToolTest, TestReloadAndDeleteDocs, TestPivotHelperCode, SolrCmdDistributorTest, SolrSlf4jReporterTest, MoveReplicaHDFSFailoverTest, RequestHandlersTest, ZkFailoverTest, TestQuerySenderListener, FacetPivotSmallTest, BasicDistributedZk2Test, ResponseBuilderTest, TestLuceneMatchVersion, MetricUtilsTest, TestDistributedMissingSort, BooleanFieldTest, TestFieldSortValues, HttpTriggerListenerTest, OverseerRolesTest, SortSpecParsingTest, SegmentsInfoRequestHandlerTest, RegexBoostProcessorTest, ZkControllerTest, MetricTriggerIntegrationTest, CleanupOldIndexTest, TestCollationFieldDocValues, ExecutePlanActionTest, TestNumericTerms32, SliceStateTest, CoreAdminHandlerTest, TestEmbeddedSolrServerConstructors, BigEndianAscendingWordDeserializerTest, HdfsUnloadDistributedZkTest, PKIAuthenticationIntegrationTest, GraphQueryTest, TestPolicyCloud, BlockCacheTest, TestSolrCoreSnapshots, TestReversedWildcardFilterFactory, BufferStoreTest, TriggerIntegrationTest, LegacyCloudClusterPropTest, CircularListTest, PrimitiveFieldTypeTest, TestBadConfig, TestRandomFaceting, ReplicationFactorTest, TestFieldCacheSortRandom, ResourceLoaderTest, StressHdfsTest, RollingRestartTest, TestCloudManagedSchema, DebugComponentTest, TestSQLHandlerNonCloud, TestWriterPerf, ConfigSetsAPITest, TestSkipOverseerOperations, TestSubQueryTransformerDistrib, TestDynamicFieldCollectionResource, BlockJoinFacetRandomTest, TemplateUpdateProcessorTest, TestDocSet, MergeStrategyTest, TestNumericRangeQuery32, TestExtendedDismaxParser, ResponseLogComponentTest, TaggerTest, DistributedTermsComponentTest, JavabinLoaderTest, ShardRoutingTest, CreateCollectionCleanupTest, CoreAdminOperationTest, TestCloudInspectUtil, TestSmileRequest, JsonLoaderTest, TestCoreDiscovery, TestCloudPseudoReturnFields, TestLegacyFieldReuse, TimeZoneUtilsTest, TestCloudSearcherWarming, DistributedFacetPivotSmallAdvancedTest, PeerSyncTest, HighlighterConfigTest, EchoParamsTest, TestWordDelimiterFilterFactory, TestSchemaNameResource, TestMaxScoreQueryParser, TestCloudConsistency, TestHdfsCloudBackupRestore, TestSSLRandomization, TestSolrCoreParser, TestSolrXml, TestAtomicUpdateErrorCases, ManagedSchemaRoundRobinCloudTest, NoCacheHeaderTest, ZkSolrClientTest, RulesTest, TestUnifiedSolrHighlighter, NodeAddedTriggerIntegrationTest, SpellPossibilityIteratorTest, TimeRoutedAliasUpdateProcessorTest, TestCloudJSONFacetSKG, TestCrossCoreJoin, DistributedVersionInfoTest, TestXmlQParserPlugin, MetricsHandlerTest, TestStreamBody, TestSolrCoreProperties, SyncSliceTest, TestCollectionAPI, AutoScalingHandlerTest, TestExpandComponent, TestMacroExpander, IndexSchemaTest, CollectionTooManyReplicasTest, TestLegacyTerms, ChaosMonkeyNothingIsSafeTest, ScheduledTriggerTest, UUIDFieldTest, CollectionsAPISolrJTest, TestLMDirichletSimilarityFactory, DistributedQueryComponentOptimizationTest, DocValuesMissingTest, ShardSplitTest, ForceLeaderTest, TestUpdate, LoggingHandlerTest, DirectoryFactoryTest, TestJettySolrRunner, TestNodeAddedTrigger, MinimalSchemaTest, ClassificationUpdateProcessorTest, TestInPlaceUpdatesDistrib, BJQParserTest, TestDocTermOrdsUninvertLimit, TestScoreJoinQPScore, PreAnalyzedUpdateProcessorTest, TestAnalyzedSuggestions, HdfsRestartWhileUpdatingTest, TestBackupRepositoryFactory, TestInitQParser, ReplaceNodeNoTargetTest, TestExportWriter, TestSolrJ, ClusterStateUpdateTest, TestPullReplica, TestHashPartitioner, CollectionPropsTest, TestImplicitCoreProperties, DistributedSpellCheckComponentTest, HttpPartitionOnCommitTest, TestSimpleTrackingShardHandler, TestConfigSetImmutable, ResponseHeaderTest, URLClassifyProcessorTest, TestFieldTypeCollectionResource, JSONWriterTest, SolrGangliaReporterTest, TestConfigSets, TestNestedDocsSort, TestStressCloudBlindAtomicUpdates, DocumentAnalysisRequestHandlerTest, BasicAuthIntegrationTest, ReturnFieldsTest, FileBasedSpellCheckerTest, TestRTimerTree, DirectUpdateHandlerTest, SolrCoreCheckLockOnStartupTest, TestInitParams, TestSearchPerf, TestClassicSimilarityFactory, SaslZkACLProviderTest, SimpleFacetsTest, TestMultiWordSynonyms, AnalysisErrorHandlingTest, ClassificationUpdateProcessorFactoryTest, BasicFunctionalityTest, DocValuesTest, FieldAnalysisRequestHandlerTest, MultiThreadedOCPTest, TestSchemaManager, HdfsDirectoryFactoryTest, TestReplicationHandlerBackup]
   [junit4] Completed [507/821 (1!)] on J1 in 8.82s, 2 tests, 1 error <<< FAILURES!

[...truncated 2004 lines...]
   [junit4] Suite: org.apache.solr.common.util.TestTimeSource
   [junit4]   2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-solrj\test\J1\temp\solr.common.util.TestTimeSource_16930F028652EAED-001\init-core-data-001
   [junit4]   2> 145627 WARN  (SUITE-TestTimeSource-seed#[16930F028652EAED]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=8 numCloses=8
   [junit4]   2> 145627 INFO  (SUITE-TestTimeSource-seed#[16930F028652EAED]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
   [junit4]   2> 145629 INFO  (SUITE-TestTimeSource-seed#[16930F028652EAED]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
   [junit4]   2> 145629 INFO  (SUITE-TestTimeSource-seed#[16930F028652EAED]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 145631 INFO  (TEST-TestTimeSource.testEpochTime-seed#[16930F028652EAED]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testEpochTime
   [junit4]   2> 155704 INFO  (TEST-TestTimeSource.testEpochTime-seed#[16930F028652EAED]) [    ] o.a.s.SolrTestCaseJ4 ###Ending testEpochTime
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestTimeSource -Dtests.method=testEpochTime -Dtests.seed=16930F028652EAED -Dtests.slow=true -Dtests.locale=yue -Dtests.timezone=MST -Dtests.asserts=true -Dtests.file.encoding=Cp1252
   [junit4] FAILURE 10.1s J1 | TestTimeSource.testEpochTime <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: SimTimeSource:50.0 time diff=14545000
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([16930F028652EAED:2EFF7C27128248AB]:0)
   [junit4]    > 	at org.apache.solr.common.util.TestTimeSource.doTestEpochTime(TestTimeSource.java:52)
   [junit4]    > 	at org.apache.solr.common.util.TestTimeSource.testEpochTime(TestTimeSource.java:32)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-solrj\test\J1\temp\solr.common.util.TestTimeSource_16930F028652EAED-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1121, maxMBSortInHeap=7.746602412906617, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@5bd85ad4), locale=yue, timezone=MST
   [junit4]   2> NOTE: Windows 10 10.0 amd64/Oracle Corporation 10 (64-bit)/cpus=3,threads=1,free=168194592,total=275251200
   [junit4]   2> NOTE: All tests run in this JVM: [TestDelegationTokenResponse, TestCollectionAdminRequest, UuidEvaluatorTest, SchemaTest, TestNamedListCodec, SolrExampleStreamingBinaryTest, AddEvaluatorTest, TestRetryUtil, ConcatOperationTest, TestLang, TestZkConfigManager, StreamDecoratorTest, HttpSolrClientConPoolTest, TestTimeSource]
   [junit4] Completed [89/154 (1!)] on J1 in 10.11s, 1 test, 1 failure <<< FAILURES!

[...truncated 43023 lines...]
[repro] Jenkins log URL: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/7379/consoleText

[repro] Revision: 3b9d3a760a432b97aad2c08b2f778fa2344eb14a

[repro] Ant options: "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC"

[repro] ant clean

[...truncated 6 lines...]
[repro] Test suites by module:
[repro]    solr\solrj
[repro]       TestTimeSource
[repro]    solr\core
[repro]       TestReplicationHandlerBackup
[repro] ant compile-test

[...truncated 2476 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestTimeSource" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC" -Dtests.seed=16930F028652EAED -Dtests.slow=true -Dtests.locale=yue -Dtests.timezone=MST -Dtests.asserts=true -Dtests.file.encoding=Cp1252

[...truncated 74 lines...]
[repro] ant compile-test

[...truncated 1336 lines...]
[repro] ant test-nocompile -Dtests.dups=5 -Dtests.maxfailures=5 -Dtests.class="*.TestReplicationHandlerBackup" -Dtests.showOutput=onerror "-Dargs=-XX:+UseCompressedOops -XX:+UseParallelGC" -Dtests.seed=78CC2F61B83F28D7 -Dtests.slow=true -Dtests.locale=eu -Dtests.timezone=America/North_Dakota/Center -Dtests.asserts=true -Dtests.file.encoding=UTF-8

[...truncated 79 lines...]
[repro] Failures:
[repro]   0/5 failed: org.apache.solr.common.util.TestTimeSource
[repro]   0/5 failed: org.apache.solr.handler.TestReplicationHandlerBackup
[repro] Exiting with code 0

[...truncated 75 lines...]