You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/11/16 01:07:47 UTC

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-10.0.1) - Build # 23217 - Still Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23217/
Java: 64bit/jdk-10.0.1 -XX:+UseCompressedOops -XX:+UseParallelGC

5 tests failed.
FAILED:  org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRouting

Error Message:
Error from server at http://127.0.0.1:42821/solr/collection1_shard2_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/collection1_shard2_replica_n2/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/collection1_shard2_replica_n2/update. Reason: <pre>    Can not find: /solr/collection1_shard2_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>  </body> </html> 

Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at http://127.0.0.1:42821/solr/collection1_shard2_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/collection1_shard2_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/collection1_shard2_replica_n2/update. Reason:
<pre>    Can not find: /solr/collection1_shard2_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at __randomizedtesting.SeedInfo.seed([D5624284C3FBCF8C:17D57EECC0BB3FF4]:0)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:949)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
	at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
	at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:237)
	at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRouting(CloudSolrClientTest.java:269)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42821/solr/collection1_shard2_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/collection1_shard2_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/collection1_shard2_replica_n2/update. Reason:
<pre>    Can not find: /solr/collection1_shard2_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.11.v20180605</a><hr/>

</body>
</html>

	at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
	at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
	at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
	at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	... 1 more


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:36241/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:36241/solr
	at __randomizedtesting.SeedInfo.seed([D8EC115E7D9FF1B9:191C68F250CF3B1E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:42229/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:42229/solr
	at __randomizedtesting.SeedInfo.seed([D8EC115E7D9FF1B9:191C68F250CF3B1E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:42643/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:42643/solr
	at __randomizedtesting.SeedInfo.seed([D8EC115E7D9FF1B9:191C68F250CF3B1E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:35449/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:35449/solr
	at __randomizedtesting.SeedInfo.seed([D8EC115E7D9FF1B9:191C68F250CF3B1E]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 14500 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestTlogReplica
   [junit4]   2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_D8EC115E7D9FF1B9-001/init-core-data-001
   [junit4]   2> 2004872 INFO  (SUITE-TestTlogReplica-seed#[D8EC115E7D9FF1B9]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_D8EC115E7D9FF1B9-001/tempDir-001
   [junit4]   2> 2004872 INFO  (SUITE-TestTlogReplica-seed#[D8EC115E7D9FF1B9]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2004872 INFO  (Thread-5945) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2004872 INFO  (Thread-5945) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2004879 ERROR (Thread-5945) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2004972 INFO  (SUITE-TestTlogReplica-seed#[D8EC115E7D9FF1B9]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:34105
   [junit4]   2> 2004976 INFO  (zkConnectionManagerCallback-9111-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2004982 WARN  (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [    ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10065783af90000, likely client has closed socket
   [junit4]   2> 2004985 INFO  (jetty-launcher-9108-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 2004985 INFO  (jetty-launcher-9108-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 10.0.1+10
   [junit4]   2> 2004986 INFO  (jetty-launcher-9108-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2004986 INFO  (jetty-launcher-9108-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2004986 INFO  (jetty-launcher-9108-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2004987 INFO  (jetty-launcher-9108-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@316c1b30{/solr,null,AVAILABLE}
   [junit4]   2> 2004988 INFO  (jetty-launcher-9108-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@336c2dcc{SSL,[ssl, http/1.1]}{127.0.0.1:43567}
   [junit4]   2> 2004988 INFO  (jetty-launcher-9108-thread-2) [    ] o.e.j.s.Server Started @2005008ms
   [junit4]   2> 2004988 INFO  (jetty-launcher-9108-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43567}
   [junit4]   2> 2004988 ERROR (jetty-launcher-9108-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2004988 INFO  (jetty-launcher-9108-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2004988 INFO  (jetty-launcher-9108-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2004989 INFO  (jetty-launcher-9108-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2004989 INFO  (jetty-launcher-9108-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2004989 INFO  (jetty-launcher-9108-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-11-16T00:28:46.001074Z
   [junit4]   2> 2004990 INFO  (jetty-launcher-9108-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2004990 INFO  (jetty-launcher-9108-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2004990 INFO  (jetty-launcher-9108-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2004990 INFO  (jetty-launcher-9108-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@55e70243{/solr,null,AVAILABLE}
   [junit4]   2> 2004991 INFO  (zkConnectionManagerCallback-9113-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2004991 INFO  (jetty-launcher-9108-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@2f88cb90{SSL,[ssl, http/1.1]}{127.0.0.1:35449}
   [junit4]   2> 2004991 INFO  (jetty-launcher-9108-thread-1) [    ] o.e.j.s.Server Started @2005011ms
   [junit4]   2> 2004991 INFO  (jetty-launcher-9108-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35449}
   [junit4]   2> 2004991 ERROR (jetty-launcher-9108-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2004991 INFO  (jetty-launcher-9108-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2004991 INFO  (jetty-launcher-9108-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr™ version 8.0.0
   [junit4]   2> 2004991 INFO  (jetty-launcher-9108-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2004991 INFO  (jetty-launcher-9108-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2004991 INFO  (jetty-launcher-9108-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-11-16T00:28:46.003697Z
   [junit4]   2> 2004991 INFO  (jetty-launcher-9108-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2004993 INFO  (zkConnectionManagerCallback-9115-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2004994 INFO  (jetty-launcher-9108-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2005097 INFO  (jetty-launcher-9108-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34105/solr
   [junit4]   2> 2005109 INFO  (zkConnectionManagerCallback-9119-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2005113 INFO  (zkConnectionManagerCallback-9121-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2005244 INFO  (jetty-launcher-9108-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34105/solr
   [junit4]   2> 2005250 INFO  (zkConnectionManagerCallback-9127-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2005265 INFO  (zkConnectionManagerCallback-9129-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2005310 INFO  (jetty-launcher-9108-thread-2) [n:127.0.0.1:43567_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43567_solr
   [junit4]   2> 2005313 INFO  (jetty-launcher-9108-thread-2) [n:127.0.0.1:43567_solr    ] o.a.s.c.Overseer Overseer (id=72169161097805830-127.0.0.1:43567_solr-n_0000000000) starting
   [junit4]   2> 2005334 INFO  (zkConnectionManagerCallback-9136-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2005342 INFO  (jetty-launcher-9108-thread-2) [n:127.0.0.1:43567_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34105/solr ready
   [junit4]   2> 2005355 INFO  (jetty-launcher-9108-thread-2) [n:127.0.0.1:43567_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43567_solr
   [junit4]   2> 2005356 DEBUG (OverseerAutoScalingTriggerThread-72169161097805830-127.0.0.1:43567_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
   [junit4]   2> 2005360 DEBUG (OverseerAutoScalingTriggerThread-72169161097805830-127.0.0.1:43567_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
   [junit4]   2> 2005361 DEBUG (OverseerAutoScalingTriggerThread-72169161097805830-127.0.0.1:43567_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
   [junit4]   2> 2005361 DEBUG (OverseerAutoScalingTriggerThread-72169161097805830-127.0.0.1:43567_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
   [junit4]   2> 2005361 INFO  (zkCallback-9128-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2005364 INFO  (zkCallback-9120-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2005374 INFO  (zkCallback-9135-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2005378 INFO  (jetty-launcher-9108-thread-1) [n:127.0.0.1:35449_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2005378 INFO  (jetty-launcher-9108-thread-1) [n:127.0.0.1:35449_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35449_solr
   [junit4]   2> 2005385 INFO  (OverseerStateUpdate-72169161097805830-127.0.0.1:43567_solr-n_0000000000) [n:127.0.0.1:43567_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:43567_solr
   [junit4]   2> 2005415 INFO  (zkCallback-9128-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2005414 INFO  (jetty-launcher-9108-thread-2) [n:127.0.0.1:43567_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2005415 DEBUG (OverseerAutoScalingTriggerThread-72169161097805830-127.0.0.1:43567_solr-n_0000000000) [    ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:43567_solr, 127.0.0.1:35449_solr]
   [junit4]   2> 2005407 INFO  (zkCallback-9135-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2005417 DEBUG (OverseerAutoScalingTriggerThread-72169161097805830-127.0.0.1:43567_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
   [junit4]   2> 2005417 DEBUG (OverseerAutoScalingTriggerThread-72169161097805830-127.0.0.1:43567_solr-n_0000000000) [    ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
   [junit4]   2> 2005408 INFO  (zkCallback-9120-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2005432 INFO  (jetty-launcher-9108-thread-2) [n:127.0.0.1:43567_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43567.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79390413
   [junit4]   2> 2005440 INFO  (jetty-launcher-9108-thread-2) [n:127.0.0.1:43567_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43567.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79390413
   [junit4]   2> 2005440 INFO  (jetty-launcher-9108-thread-2) [n:127.0.0.1:43567_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43567.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79390413
   [junit4]   2> 2005440 INFO  (jetty-launcher-9108-thread-2) [n:127.0.0.1:43567_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_D8EC115E7D9FF1B9-001/tempDir-001/node2/.
   [junit4]   2> 2005445 DEBUG (ScheduledTrigger-9134-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2005452 INFO  (zkConnectionManagerCallback-9142-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2005460 INFO  (jetty-launcher-9108-thread-1) [n:127.0.0.1:35449_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2005463 INFO  (jetty-launcher-9108-thread-1) [n:127.0.0.1:35449_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34105/solr ready
   [junit4]   2> 2005476 INFO  (jetty-launcher-9108-thread-1) [n:127.0.0.1:35449_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2005490 INFO  (jetty-launcher-9108-thread-1) [n:127.0.0.1:35449_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35449.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79390413
   [junit4]   2> 2005503 INFO  (jetty-launcher-9108-thread-1) [n:127.0.0.1:35449_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35449.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79390413
   [junit4]   2> 2005503 INFO  (jetty-launcher-9108-thread-1) [n:127.0.0.1:35449_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35449.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79390413
   [junit4]   2> 2005504 INFO  (jetty-launcher-9108-thread-1) [n:127.0.0.1:35449_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_D8EC115E7D9FF1B9-001/tempDir-001/node1/.
   [junit4]   2> 2005529 INFO  (zkConnectionManagerCallback-9145-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2005540 INFO  (zkConnectionManagerCallback-9150-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2005541 INFO  (SUITE-TestTlogReplica-seed#[D8EC115E7D9FF1B9]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2005542 INFO  (SUITE-TestTlogReplica-seed#[D8EC115E7D9FF1B9]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34105/solr ready
   [junit4]   2> 2005544 INFO  (SUITE-TestTlogReplica-seed#[D8EC115E7D9FF1B9]-worker) [    ] o.a.s.c.TestTlogReplica Using legacyCloud?: false
   [junit4]   2> 2005557 INFO  (qtp1241360304-30103) [n:127.0.0.1:43567_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2005558 INFO  (qtp1241360304-30103) [n:127.0.0.1:43567_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2005573 INFO  (qtp1241360304-30105) [n:127.0.0.1:43567_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_only_leader_indexes&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2005575 INFO  (OverseerThreadFactory-9136-thread-1-processing-n:127.0.0.1:43567_solr) [n:127.0.0.1:43567_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection tlog_replica_test_only_leader_indexes
   [junit4]   2> 2005681 INFO  (qtp1241360304-30101) [n:127.0.0.1:43567_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2005695 INFO  (qtp1006846948-30109) [n:127.0.0.1:35449_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2005700 INFO  (OverseerStateUpdate-72169161097805830-127.0.0.1:43567_solr-n_0000000000) [n:127.0.0.1:43567_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"tlog_replica_test_only_leader_indexes",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"tlog_replica_test_only_leader_indexes_shard1_replica_t1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:35449/solr",
   [junit4]   2>   "type":"TLOG",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2005700 INFO  (OverseerStateUpdate-72169161097805830-127.0.0.1:43567_solr-n_0000000000) [n:127.0.0.1:43567_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"tlog_replica_test_only_leader_indexes",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"tlog_replica_test_only_leader_indexes_shard1_replica_t2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:43567/solr",
   [junit4]   2>   "type":"TLOG",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2005906 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr    x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_only_leader_indexes_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_only_leader_indexes&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 2005906 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr    x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=tlog_replica_test_only_leader_indexes_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_only_leader_indexes&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 2005906 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr    x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2006454 DEBUG (ScheduledTrigger-9134-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2006933 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2006935 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
   [junit4]   2> 2006936 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.IndexSchema [tlog_replica_test_only_leader_indexes_shard1_replica_t1] Schema name=minimal
   [junit4]   2> 2006938 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2006938 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_only_leader_indexes_shard1_replica_t1' using configuration from collection tlog_replica_test_only_leader_indexes, trusted=true
   [junit4]   2> 2006938 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35449.solr.core.tlog_replica_test_only_leader_indexes.shard1.replica_t1' (registry 'solr.core.tlog_replica_test_only_leader_indexes.shard1.replica_t1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79390413
   [junit4]   2> 2006939 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SolrCore [[tlog_replica_test_only_leader_indexes_shard1_replica_t1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_D8EC115E7D9FF1B9-001/tempDir-001/node1/tlog_replica_test_only_leader_indexes_shard1_replica_t1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_D8EC115E7D9FF1B9-001/tempDir-001/node1/./tlog_replica_test_only_leader_indexes_shard1_replica_t1/data/]
   [junit4]   2> 2006940 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.s.IndexSchema [tlog_replica_test_only_leader_indexes_shard1_replica_t2] Schema name=minimal
   [junit4]   2> 2006942 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2006942 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_only_leader_indexes_shard1_replica_t2' using configuration from collection tlog_replica_test_only_leader_indexes, trusted=true
   [junit4]   2> 2006946 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43567.solr.core.tlog_replica_test_only_leader_indexes.shard1.replica_t2' (registry 'solr.core.tlog_replica_test_only_leader_indexes.shard1.replica_t2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79390413
   [junit4]   2> 2006946 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.SolrCore [[tlog_replica_test_only_leader_indexes_shard1_replica_t2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_D8EC115E7D9FF1B9-001/tempDir-001/node2/tlog_replica_test_only_leader_indexes_shard1_replica_t2], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_D8EC115E7D9FF1B9-001/tempDir-001/node2/./tlog_replica_test_only_leader_indexes_shard1_replica_t2/data/]
   [junit4]   2> 2006988 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2006988 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2006989 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2006989 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2006990 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2037e83c[tlog_replica_test_only_leader_indexes_shard1_replica_t1] main]
   [junit4]   2> 2006991 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2006991 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2006991 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2006991 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1617248259148873728
   [junit4]   2> 2006993 INFO  (searcherExecutor-9145-thread-1-processing-n:127.0.0.1:35449_solr x:tlog_replica_test_only_leader_indexes_shard1_replica_t1 c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_only_leader_indexes_shard1_replica_t1] Registered new searcher Searcher@2037e83c[tlog_replica_test_only_leader_indexes_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2006995 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_only_leader_indexes/terms/shard1 to Terms{values={core_node3=0}, version=0}
   [junit4]   2> 2006997 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
   [junit4]   2> 2006999 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2006999 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2007000 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2007000 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2007000 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@4dce0c09[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main]
   [junit4]   2> 2007001 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2007002 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2007002 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2007002 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1617248259160408064
   [junit4]   2> 2007003 INFO  (searcherExecutor-9146-thread-1-processing-n:127.0.0.1:43567_solr x:tlog_replica_test_only_leader_indexes_shard1_replica_t2 c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_only_leader_indexes_shard1_replica_t2] Registered new searcher Searcher@4dce0c09[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2007016 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_only_leader_indexes/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
   [junit4]   2> 2007454 DEBUG (ScheduledTrigger-9134-thread-4) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2007501 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2007501 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2007501 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:35449/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/
   [junit4]   2> 2007501 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_only_leader_indexes_shard1_replica_t1 url=https://127.0.0.1:35449/solr START replicas=[https://127.0.0.1:43567/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t2/] nUpdates=100
   [junit4]   2> 2007501 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_only_leader_indexes_shard1_replica_t1 url=https://127.0.0.1:35449/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 2007504 INFO  (qtp1241360304-30178) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2007504 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 2007504 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2007504 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ZkController tlog_replica_test_only_leader_indexes_shard1_replica_t1 stopping background replication from leader
   [junit4]   2> 2007512 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:35449/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/ shard1
   [junit4]   2> 2007613 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2007714 INFO  (zkCallback-9120-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2007759 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_only_leader_indexes_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_only_leader_indexes&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1853
   [junit4]   2> 2008018 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.ZkController tlog_replica_test_only_leader_indexes_shard1_replica_t2 starting background replication from leader
   [junit4]   2> 2008019 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:01
   [junit4]   2> 2008019 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
   [junit4]   2> 2008019 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2008020 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Updated masterUrl to https://127.0.0.1:35449/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/
   [junit4]   2> 2008021 INFO  (qtp1006846948-30114) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 2008021 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 2008021 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 2008021 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 2008021 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 2008021 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
   [junit4]   2> 2008021 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@7ecf2f80[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main]
   [junit4]   2> 2008023 INFO  (searcherExecutor-9146-thread-1) [    ] o.a.s.c.SolrCore [tlog_replica_test_only_leader_indexes_shard1_replica_t2] Registered new searcher Searcher@7ecf2f80[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2008025 INFO  (qtp1241360304-30107) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=tlog_replica_test_only_leader_indexes_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_only_leader_indexes&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2119
   [junit4]   2> 2008027 INFO  (qtp1241360304-30105) [n:127.0.0.1:43567_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2008121 INFO  (zkCallback-9128-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2008121 INFO  (zkCallback-9120-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_only_leader_indexes/state.json] for collection [tlog_replica_test_only_leader_indexes] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2008454 DEBUG (ScheduledTrigger-9134-thread-2) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2009020 INFO  (qtp1006846948-30114) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 2009020 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 2009020 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 2009020 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 2009020 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 2009020 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
   [junit4]   2> 2009020 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@93312f3[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main]
   [junit4]   2> 2009021 INFO  (searcherExecutor-9146-thread-1) [    ] o.a.s.c.SolrCore [tlog_replica_test_only_leader_indexes_shard1_replica_t2] Registered new searcher Searcher@93312f3[tlog_replica_test_only_leader_indexes_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2009027 INFO  (qtp1241360304-30105) [n:127.0.0.1:43567_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_only_leader_indexes&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2} status=0 QTime=3453
   [junit4]   2> 2009035 INFO  (qtp1006846948-30114) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_only_leader_indexes/terms/shard1 to Terms{values={core_node3=1, core_node4=1}, version=2}
   [junit4]   2> 2009051 INFO  (qtp1241360304-30105) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:35449/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/&wt=javabin&version=2}{add=[1 (1617248261291114496), 2 (1617248261291114497), 3 (1617248261292163072), 4 (1617248261292163073)]} 0 10
   [junit4]   2> 2009052 INFO  (qtp1006846948-30114) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1 (1617248261291114496), 2 (1617248261291114497), 3 (1617248261292163072), 4 (1617248261292163073)]} 0 18
   [junit4]   2> 2009062 INFO  (qtp1241360304-30105) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/get params={distrib=false&qt=/get&id=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2009062 INFO  (qtp1241360304-30173) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/get params={distrib=false&qt=/get&id=2&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2009063 INFO  (qtp1241360304-30105) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/get params={distrib=false&qt=/get&id=3&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2009063 INFO  (qtp1241360304-30173) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/get params={distrib=false&qt=/get&id=4&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2009070 INFO  (qtp1006846948-30114) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/get params={distrib=false&qt=/get&id=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2009070 INFO  (qtp1006846948-30104) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/get params={distrib=false&qt=/get&id=2&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2009071 INFO  (qtp1006846948-30114) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/get params={distrib=false&qt=/get&id=3&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2009071 INFO  (qtp1006846948-30104) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/get params={distrib=false&qt=/get&id=4&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2009073 INFO  (qtp1241360304-30103) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:35449/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/&wt=javabin&version=2}{delete=[1 (-1617248261330960384)]} 0 0
   [junit4]   2> 2009073 INFO  (qtp1006846948-30177) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/update params={wt=javabin&version=2}{delete=[1 (-1617248261330960384)]} 0 1
   [junit4]   2> 2009076 INFO  (qtp1006846948-30104) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@70948cef[tlog_replica_test_only_leader_indexes_shard1_replica_t1] realtime]
   [junit4]   2> 2009077 INFO  (qtp1006846948-30104) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3ea77691[tlog_replica_test_only_leader_indexes_shard1_replica_t1] realtime]
   [junit4]   2> 2009078 INFO  (qtp1241360304-30105) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&_version_=-1617248261333057536&distrib.from=https://127.0.0.1:35449/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/&wt=javabin&version=2}{deleteByQuery=id:2 (-1617248261333057536)} 0 0
   [junit4]   2> 2009078 INFO  (qtp1006846948-30104) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=id:2 (-1617248261333057536)} 0 4
   [junit4]   2> 2009083 INFO  (qtp1241360304-30105) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/get params={distrib=false&qt=/get&id=2&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2009083 INFO  (qtp1241360304-30101) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/get params={distrib=false&qt=/get&id=3&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2009084 INFO  (qtp1241360304-30105) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/get params={distrib=false&qt=/get&id=4&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2009087 INFO  (qtp1006846948-30106) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1617248261346689024,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 2009087 INFO  (qtp1006846948-30106) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@2b45b229 commitCommandVersion:1617248261346689024
   [junit4]   2> 2009089 INFO  (qtp1006846948-30106) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3df4446d[tlog_replica_test_only_leader_indexes_shard1_replica_t1] main]
   [junit4]   2> 2009089 INFO  (qtp1241360304-30103) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:35449/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
   [junit4]   2> 2009089 INFO  (qtp1006846948-30106) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 2009090 INFO  (searcherExecutor-9145-thread-1-processing-n:127.0.0.1:35449_solr x:tlog_replica_test_only_leader_indexes_shard1_replica_t1 c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_only_leader_indexes_shard1_replica_t1] Registered new searcher Searcher@3df4446d[tlog_replica_test_only_leader_indexes_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader(_1(8.0.0):C2))}
   [junit4]   2> 2009090 INFO  (qtp1006846948-30106) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:35449/solr/tlog_replica_test_only_leader_indexes_shard1_replica_t1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 3
   [junit4]   2> 2009090 INFO  (qtp1006846948-30177) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/update params={_stateVer_=tlog_replica_test_only_leader_indexes:6&commit=true&wt=javabin&version=2}{commit=} 0 5
   [junit4]   2> 2009094 INFO  (qtp1006846948-30177) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=2 status=0 QTime=0
   [junit4]   2> 2009096 INFO  (qtp1241360304-30103) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2009197 INFO  (qtp1241360304-30178) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2009298 INFO  (qtp1241360304-30103) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2009399 INFO  (qtp1241360304-30178) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2009455 DEBUG (ScheduledTrigger-9134-thread-3) [    ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
   [junit4]   2> 2009500 INFO  (qtp1241360304-30103) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2009577 INFO  (OverseerCollectionConfigSetProcessor-72169161097805830-127.0.0.1:43567_solr-n_0000000000) [n:127.0.0.1:43567_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 2009605 INFO  (qtp1241360304-30103) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2009706 INFO  (qtp1241360304-30178) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2009806 INFO  (qtp1241360304-30103) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2009907 INFO  (qtp1241360304-30178) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2010008 INFO  (qtp1241360304-30103) [n:127.0.0.1:43567_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node4 x:tlog_replica_test_only_leader_indexes_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 2010020 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 2010024 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 2
   [junit4]   2> 2010024 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 1542328130099
   [junit4]   2> 2010024 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 2010024 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 2010024 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Starting replication process
   [junit4]   2> 2010025 INFO  (qtp1006846948-30109) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&tlogFiles=false&wt=javabin&version=2&command=filelist} status=0 QTime=0
   [junit4]   2> 2010025 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Number of files in latest index in master: 13
   [junit4]   2> 2010026 INFO  (indexFetcher-9156-thread-1) [    ] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@520926d3 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@18c903a9)
   [junit4]   2> 2010027 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_1.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 2010040 INFO  (qtp1006846948-30109) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_1_Lucene70_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 2010041 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_1.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 2010042 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_1.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 2010043 INFO  (qtp1006846948-30109) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_1.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 2010044 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_1_Lucene50_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 2010045 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_1_Lucene50_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 2010046 INFO  (qtp1006846948-30109) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_1_Lucene50_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 2010046 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_1.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 2010047 INFO  (qtp1006846948-30109) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_1.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 2010048 INFO  (qtp1006846948-30111) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes s:shard1 r:core_node3 x:tlog_replica_test_only_leader_indexes_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_only_leader_indexes_shard1_replica_t1]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_1_Lucene50_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 2010049 INFO  (qtp1006846948-30109) [n:127.0.0.1:35449_solr c:tlog_replica_test_only_leader_indexes

[...truncated too long message...]

> 195394 INFO  (zkCallback-236-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 195395 INFO  (jetty-closer-134-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72169274095763481-127.0.0.1:45087_solr-n_0000000005) closing
   [junit4]   2> 195396 INFO  (OverseerStateUpdate-72169274095763481-127.0.0.1:45087_solr-n_0000000005) [n:127.0.0.1:45087_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:45087_solr
   [junit4]   2> 195396 WARN  (OverseerAutoScalingTriggerThread-72169274095763481-127.0.0.1:45087_solr-n_0000000005) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 195398 INFO  (jetty-closer-134-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4aeeab18{/solr,null,UNAVAILABLE}
   [junit4]   2> 195398 INFO  (zkCallback-253-thread-2) [    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42643_solr
   [junit4]   2> 195398 INFO  (jetty-closer-134-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 195399 INFO  (jetty-closer-134-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 195399 INFO  (jetty-closer-134-thread-2) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@254732fd: rootName = solr_42643, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@721e18b3
   [junit4]   2> 195399 INFO  (jetty-closer-134-thread-2) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:42643_solr
   [junit4]   2> 195400 INFO  (jetty-closer-134-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 195400 INFO  (zkCallback-253-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 195400 INFO  (zkCallback-260-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 195403 INFO  (jetty-closer-134-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@608860ac{/solr,null,UNAVAILABLE}
   [junit4]   2> 195403 INFO  (jetty-closer-134-thread-2) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 195404 ERROR (SUITE-TestTlogReplica-seed#[D8EC115E7D9FF1B9]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 195405 INFO  (SUITE-TestTlogReplica-seed#[D8EC115E7D9FF1B9]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45095 45095
   [junit4]   2> 195513 INFO  (Thread-103) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45095 45095
   [junit4]   2> 195513 WARN  (Thread-103) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	32	/solr/configs/conf
   [junit4]   2> 	32	/solr/collections/tlog_replica_test_create_delete/terms/shard1
   [junit4]   2> 	30	/solr/collections/tlog_replica_test_create_delete/terms/shard2
   [junit4]   2> 	19	/solr/collections/tlog_replica_test_remove_leader/terms/shard1
   [junit4]   2> 	18	/solr/collections/tlog_replica_test_recovery/terms/shard1
   [junit4]   2> 	17	/solr/collections/tlog_replica_test_kill_leader/terms/shard1
   [junit4]   2> 	17	/solr/collections/tlog_replica_test_kill_tlog_replica/terms/shard1
   [junit4]   2> 	16	/solr/collections/tlog_replica_test_basic_leader_election/terms/shard1
   [junit4]   2> 	16	/solr/collections/tlog_replica_test_real_time_get/terms/shard1
   [junit4]   2> 	15	/solr/aliases.json
   [junit4]   2> 	14	/solr/collections/tlog_replica_test_add_docs/terms/shard1
   [junit4]   2> 	10	/solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/terms/shard1
   [junit4]   2> 	10	/solr/collections/tlog_replica_test_delete_by_id/terms/shard1
   [junit4]   2> 	10	/solr/collections/tlog_replica_test_only_leader_indexes/terms/shard1
   [junit4]   2> 	7	/solr/security.json
   [junit4]   2> 	6	/solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard2
   [junit4]   2> 	5	/solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	23	/solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/state.json
   [junit4]   2> 	23	/solr/collections/tlog_replica_test_create_delete/state.json
   [junit4]   2> 	15	/solr/clusterstate.json
   [junit4]   2> 	15	/solr/clusterprops.json
   [junit4]   2> 	14	/solr/collections/tlog_replica_test_kill_leader/state.json
   [junit4]   2> 	13	/solr/collections/tlog_replica_test_recovery/state.json
   [junit4]   2> 	13	/solr/collections/tlog_replica_test_basic_leader_election/state.json
   [junit4]   2> 	11	/solr/collections/tlog_replica_test_kill_tlog_replica/state.json
   [junit4]   2> 	9	/solr/collections/tlog_replica_test_remove_leader/state.json
   [junit4]   2> 	7	/solr/collections/tlog_replica_test_real_time_get/state.json
   [junit4]   2> 	5	/solr/collections/tlog_replica_test_add_docs/state.json
   [junit4]   2> 	5	/solr/autoscaling.json
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72169274095763460-core_node16-n_0000000000
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72169274095763460-core_node3-n_0000000000
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_add_remove_tlog_replica/state.json
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_only_leader_indexes/state.json
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_delete_by_id/state.json
   [junit4]   2> 	2	/solr/overseer_elect/election/72169274095763473-127.0.0.1:42643_solr-n_0000000003
   [junit4]   2> 	2	/solr/collections/tlog_replica_test_kill_tlog_replica/leader_elect/shard1/election/72169274095763473-core_node4-n_0000000000
   [junit4]   2> 	2	/solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72169274095763463-core_node5-n_0000000000
   [junit4]   2> 	2	/solr/collections/tlog_replica_test_real_time_get/leader_elect/shard1/election/72169274095763485-core_node5-n_0000000000
   [junit4]   2> 	2	/solr/collections/tlog_replica_test_recovery/leader_elect/shard1/election/72169274095763481-core_node3-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	15	/solr/live_nodes
   [junit4]   2> 	15	/solr/collections
   [junit4]   2> 	5	/solr/overseer/queue
   [junit4]   2> 	5	/solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 	5	/solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 	5	/solr/overseer/queue-work
   [junit4]   2> 	5	/solr/overseer/collection-queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestTlogReplica_D8EC115E7D9FF1B9-002
   [junit4]   2> Nov 16, 2018 12:58:58 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene80, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2170fa9c), locale=xog, timezone=Atlantic/St_Helena
   [junit4]   2> NOTE: Linux 4.15.0-38-generic amd64/Oracle Corporation 10.0.1 (64-bit)/cpus=8,threads=1,free=154455784,total=524812288
   [junit4]   2> NOTE: All tests run in this JVM: [TestTlogReplica, TestTlogReplica]
   [junit4] Completed [5/5 (5!)] on J2 in 89.63s, 13 tests, 1 failure <<< FAILURES!

[...truncated 15 lines...]
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1092: There were test failures: 5 suites, 65 tests, 5 failures [seed: D8EC115E7D9FF1B9]

Total time: 3 minutes 18 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   0/5 failed: org.apache.solr.client.solrj.impl.CloudSolrClientTest
[repro]   5/5 failed: org.apache.solr.cloud.TestTlogReplica
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=df5540acc99fe287758433701108303fedb2c5b6, workspace=/var/lib/jenkins/workspace/Lucene-Solr-master-Linux
[WARNINGS] Computing warning deltas based on reference build #23216
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-9.0.4) - Build # 23218 - Failure!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/23218/
Java: 64bit/jdk-9.0.4 -XX:-UseCompressedOops -XX:+UseSerialGC

All tests passed

Build Log:
[...truncated 15324 lines...]
   [junit4] JVM J1: stdout was not empty, see: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J1-20181116_065720_654933453222896792501.sysout
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] #
   [junit4] # A fatal error has been detected by the Java Runtime Environment:
   [junit4] #
   [junit4] #  SIGSEGV (0xb) at pc=0x00007ff09eb8fdc9, pid=23878, tid=23909
   [junit4] #
   [junit4] # JRE version: Java(TM) SE Runtime Environment (9.0+11) (build 9.0.4+11)
   [junit4] # Java VM: Java HotSpot(TM) 64-Bit Server VM (9.0.4+11, mixed mode, tiered, serial gc, linux-amd64)
   [junit4] # Problematic frame:
   [junit4] # V  [libjvm.so+0xc5adc9]  PhaseIdealLoop::split_up(Node*, Node*, Node*) [clone .part.40]+0x619
   [junit4] #
   [junit4] # No core dump will be written. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
   [junit4] #
   [junit4] # An error report file with more information is saved as:
   [junit4] # /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/hs_err_pid23878.log
   [junit4] #
   [junit4] # Compiler replay data is saved as:
   [junit4] # /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/replay_pid23878.log
   [junit4] #
   [junit4] # If you would like to submit a bug report, please visit:
   [junit4] #   http://bugreport.java.com/bugreport/crash.jsp
   [junit4] #
   [junit4] <<< JVM J1: EOF ----

[...truncated 236 lines...]
   [junit4] ERROR: JVM J1 ended with an exception, command line: /home/jenkins/tools/java/64bit/jdk-9.0.4/bin/java -XX:-UseCompressedOops -XX:+UseSerialGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/home/jenkins/workspace/Lucene-Solr-master-Linux/heapdumps -ea -esa --illegal-access=deny -Dtests.prefix=tests -Dtests.seed=AB9D6235E0F73CAE -Xmx512M -Dtests.iters= -Dtests.verbose=false -Dtests.infostream=false -Dtests.codec=random -Dtests.postingsformat=random -Dtests.docvaluesformat=random -Dtests.locale=random -Dtests.timezone=random -Dtests.directory=random -Dtests.linedocsfile=europarl.lines.txt.gz -Dtests.luceneMatchVersion=8.0.0 -Dtests.cleanthreads=perClass -Djava.util.logging.config.file=/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/tools/junit4/logging.properties -Dtests.nightly=false -Dtests.weekly=false -Dtests.monster=false -Dtests.slow=true -Dtests.asserts=true -Dtests.multiplier=3 -DtempDir=./temp -Djava.io.tmpdir=./temp -Dcommon.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene -Dclover.db.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/clover/db -Djava.security.policy=/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/tools/junit4/solr-tests.policy -Dtests.LUCENE_VERSION=8.0.0 -Djetty.testMode=1 -Djetty.insecurerandom=1 -Dsolr.directoryFactory=org.apache.solr.core.MockDirectoryFactory -Djava.awt.headless=true -Djdk.map.althashing.threshold=0 -Dtests.src.home=/home/jenkins/workspace/Lucene-Solr-master-Linux -Djava.security.egd=file:/dev/./urandom -Djunit4.childvm.cwd=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1 -Djunit4.tempDir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp -Djunit4.childvm.id=1 -Djunit4.childvm.count=3 -Djava.security.manager=org.apache.lucene.util.TestSecurityManager -Dtests.filterstacks=true -Dtests.leaveTemporary=false -Dtests.badapples=false -classpath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/classes/test:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/test-framework/lib/junit4-ant-2.6.4.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/test-framework/classes/java:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/codecs/classes/java:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-solrj/classes/java:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/classes/java:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/common/lucene-analyzers-common-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/kuromoji/lucene-analyzers-kuromoji-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/nori/lucene-analyzers-nori-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/phonetic/lucene-analyzers-phonetic-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/codecs/lucene-codecs-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/backward-codecs/lucene-backward-codecs-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/highlighter/lucene-highlighter-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/memory/lucene-memory-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/misc/lucene-misc-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/spatial-extras/lucene-spatial-extras-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/spatial3d/lucene-spatial3d-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/expressions/lucene-expressions-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/suggest/lucene-suggest-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/grouping/lucene-grouping-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/queries/lucene-queries-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/queryparser/lucene-queryparser-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/join/lucene-join-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/sandbox/lucene-sandbox-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/classification/lucene-classification-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/antlr4-runtime-4.5.1-1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/asm-5.1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/asm-commons-5.1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/avatica-core-1.10.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/caffeine-2.4.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/calcite-core-1.13.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/calcite-linq4j-1.13.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/commons-cli-1.2.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/commons-codec-1.11.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/commons-collections-3.2.2.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/commons-compiler-2.7.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/commons-configuration-1.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/commons-exec-1.3.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/commons-fileupload-1.3.3.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/commons-io-2.5.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/commons-lang-2.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/commons-lang3-3.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/commons-math3-3.6.1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/curator-client-2.8.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/curator-framework-2.8.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/curator-recipes-2.8.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/disruptor-3.4.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/dom4j-1.6.1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/eigenbase-properties-1.1.5.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/guava-14.0.1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/hadoop-annotations-2.7.4.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/hadoop-auth-2.7.4.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/hadoop-common-2.7.4.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/hadoop-hdfs-2.7.4.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/hppc-0.8.1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/htrace-core-3.2.0-incubating.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/jackson-annotations-2.9.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/jackson-core-2.9.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/jackson-core-asl-1.9.13.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/jackson-databind-2.9.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/jackson-dataformat-smile-2.9.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/jackson-mapper-asl-1.9.13.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/janino-2.7.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/jcl-over-slf4j-1.7.24.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/log4j-1.2-api-2.11.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/log4j-api-2.11.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/log4j-core-2.11.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/log4j-slf4j-impl-2.11.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/org.restlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/org.restlet.ext.servlet-2.3.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/protobuf-java-3.1.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/rrd4j-3.2.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/spatial4j-0.7.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/lib/t-digest-3.1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/solrj/lib/commons-io-2.5.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/solrj/lib/commons-math3-3.6.1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/solrj/lib/httpclient-4.5.3.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/solrj/lib/httpcore-4.4.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/solrj/lib/httpmime-4.5.3.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/solrj/lib/jcl-over-slf4j-1.7.24.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/solrj/lib/noggit-0.8.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/solrj/lib/slf4j-api-1.7.24.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/solrj/lib/stax2-api-3.1.4.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/solrj/lib/woodstox-core-asl-4.4.1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/solrj/lib/zookeeper-3.4.11.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/gmetric4j-1.0.7.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/javax.servlet-api-3.1.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-continuation-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-deploy-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-http-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-io-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-jmx-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-rewrite-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-security-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-server-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-servlet-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-servlets-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-util-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-webapp-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/jetty-xml-9.4.11.v20180605.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/metrics-core-3.2.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/metrics-ganglia-3.2.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/metrics-graphite-3.2.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/metrics-jetty9-3.2.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/server/lib/metrics-jvm-3.2.6.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/example/example-DIH/solr/db/lib/derby-10.9.1.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/example/example-DIH/solr/db/lib/hsqldb-2.4.0.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/core/classes/java:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/test-framework/lib/junit-4.10.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/test-framework/lib/randomizedtesting-runner-2.6.4.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/antlr-2.7.7.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/antlr4-runtime-4.5.1-1.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-core-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-core-api-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-core-avl-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-core-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-i18n-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/api-all-1.0.0-M20.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/bcprov-jdk15on-1.60.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/byte-buddy-1.6.2.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/commons-collections-3.2.2.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/ehcache-core-2.4.4.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/jersey-core-1.9.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/jersey-server-1.9.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/jetty-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/jetty-util-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/mina-core-2.0.0-M5.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/mockito-core-2.6.2.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/netty-all-4.0.36.Final.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/test-lib/objenesis-2.5.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/analysis/icu/lucene-analyzers-icu-8.0.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/contrib/solr-analysis-extras/classes/java:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/queryparser/classes/test:/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/build/backward-codecs/classes/test:/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/contrib/analysis-extras/lib/icu4j-62.1.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/home/jenkins/.ant/lib/ivy-2.4.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/home/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.6.4.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J1-20181116_065720_65416479443639961387829.events @/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J1-20181116_065720_65414937544951118129348.suites -stdin
   [junit4] ERROR: JVM J1 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash.  See process stdout at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J1-20181116_065720_654933453222896792501.sysout
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1542)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:123)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:997)
   [junit4] 	at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:994)
   [junit4] 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
   [junit4] 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
   [junit4] 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
   [junit4] 	at java.base/java.lang.Thread.run(Thread.java:844)

BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:633: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:577: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:59: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build.xml:267: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/common-build.xml:558: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1567: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:1092: At least one slave process threw an exception, first: Forked process returned with error code: 134. Very likely a JVM crash.  See process stdout at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/temp/junit4-J1-20181116_065720_654933453222896792501.sysout

Total time: 78 minutes 53 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2