You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/07/25 16:59:29 UTC
[JENKINS-EA] Lucene-Solr-7.x-Linux (64bit/jdk-11-ea+23) - Build #
2414 - Failure!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/2414/
Java: 64bit/jdk-11-ea+23 -XX:-UseCompressedOops -XX:+UseParallelGC
5 tests failed.
FAILED: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir
Error Message:
cluster 2 docs mismatch expected:<1000> but was:<490>
Stack Trace:
java.lang.AssertionError: cluster 2 docs mismatch expected:<1000> but was:<490>
at __randomizedtesting.SeedInfo.seed([61288D56A353D960:24F37DB4BB7D9522]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir(CdcrBidirectionalTest.java:100)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.search.mlt.CloudMLTQParserTest.testHighWLValue
Error Message:
Error from server at https://127.0.0.1:34635/solr/mlt-collection: Error from server at https://127.0.0.1:36891/solr/mlt-collection_shard1_replica_n1: Error completing MLT request. Could not fetch document with id [3]
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:34635/solr/mlt-collection: Error from server at https://127.0.0.1:36891/solr/mlt-collection_shard1_replica_n1: Error completing MLT request. Could not fetch document with id [3]
at __randomizedtesting.SeedInfo.seed([61288D56A353D960:51A6D1B01BE7D84]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.search.mlt.CloudMLTQParserTest.testHighWLValue(CloudMLTQParserTest.java:209)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: junit.framework.TestSuite.org.apache.solr.handler.TestSQLHandler
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler: 1) Thread[id=31557, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler] at java.base@11-ea/java.lang.Thread.sleep(Native Method) at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.base@11-ea/java.lang.Thread.run(Thread.java:834)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.TestSQLHandler:
1) Thread[id=31557, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestSQLHandler]
at java.base@11-ea/java.lang.Thread.sleep(Native Method)
at app//org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.base@11-ea/java.lang.Thread.run(Thread.java:834)
at __randomizedtesting.SeedInfo.seed([61288D56A353D960]:0)
FAILED: org.apache.solr.handler.TestSQLHandler.doTest
Error Message:
--> https://127.0.0.1:38555/kouia/h/collection1_shard2_replica_n41:Failed to execute sqlQuery 'select year_i, month_i, day_i, sum(item_i) from collection1 group by year_i, month_i, day_i order by year_i desc, month_i desc, day_i desc' against JDBC connection 'jdbc:calcitesolr:'. Error while executing SQL "select year_i, month_i, day_i, sum(item_i) from collection1 group by year_i, month_i, day_i order by year_i desc, month_i desc, day_i desc": java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> https://127.0.0.1:36641/kouia/h/collection1_shard1_replica_n43/:java.util.concurrent.ExecutionException: java.io.IOException: params fl=year_i,item_i,day_i,month_i&q=*:*&wt=javabin&qt=/export&partitionKeys=year_i,month_i,day_i&sort=year_i+desc,month_i+desc,day_i+desc&distrib=false
Stack Trace:
java.io.IOException: --> https://127.0.0.1:38555/kouia/h/collection1_shard2_replica_n41:Failed to execute sqlQuery 'select year_i, month_i, day_i, sum(item_i) from collection1 group by year_i, month_i, day_i order by year_i desc, month_i desc, day_i desc' against JDBC connection 'jdbc:calcitesolr:'.
Error while executing SQL "select year_i, month_i, day_i, sum(item_i) from collection1 group by year_i, month_i, day_i order by year_i desc, month_i desc, day_i desc": java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: --> https://127.0.0.1:36641/kouia/h/collection1_shard1_replica_n43/:java.util.concurrent.ExecutionException: java.io.IOException: params fl=year_i,item_i,day_i,month_i&q=*:*&wt=javabin&qt=/export&partitionKeys=year_i,month_i,day_i&sort=year_i+desc,month_i+desc,day_i+desc&distrib=false
at __randomizedtesting.SeedInfo.seed([61288D56A353D960:C66C35F2CEE8CAD9]:0)
at org.apache.solr.client.solrj.io.stream.SolrStream.read(SolrStream.java:217)
at org.apache.solr.handler.TestSQLHandler.getTuples(TestSQLHandler.java:2523)
at org.apache.solr.handler.TestSQLHandler.testParallelTimeSeriesGrouping(TestSQLHandler.java:2474)
at org.apache.solr.handler.TestSQLHandler.doTest(TestSQLHandler.java:97)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
FAILED: org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslWithCheckPeerName
Error Message:
Could not find collection:first_collection
Stack Trace:
java.lang.AssertionError: Could not find collection:first_collection
at __randomizedtesting.SeedInfo.seed([61288D56A353D960:D2AC5AA6180241B1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkCreateCollection(TestMiniSolrCloudClusterSSL.java:261)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithCollectionCreations(TestMiniSolrCloudClusterSSL.java:231)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.checkClusterWithNodeReplacement(TestMiniSolrCloudClusterSSL.java:155)
at org.apache.solr.cloud.TestMiniSolrCloudClusterSSL.testSslWithCheckPeerName(TestMiniSolrCloudClusterSSL.java:137)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:834)
Build Log:
[...truncated 12635 lines...]
[junit4] Suite: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest
[junit4] 2> 171140 INFO (SUITE-CdcrBidirectionalTest-seed#[61288D56A353D960]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/init-core-data-001
[junit4] 2> 171141 WARN (SUITE-CdcrBidirectionalTest-seed#[61288D56A353D960]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=17 numCloses=17
[junit4] 2> 171141 INFO (SUITE-CdcrBidirectionalTest-seed#[61288D56A353D960]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 171142 INFO (SUITE-CdcrBidirectionalTest-seed#[61288D56A353D960]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 171143 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBiDir
[junit4] 2> 171143 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster2-001
[junit4] 2> 171143 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 171144 INFO (Thread-271) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 171144 INFO (Thread-271) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 171150 ERROR (Thread-271) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 171244 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.ZkTestServer start zk server on port:42939
[junit4] 2> 171245 INFO (zkConnectionManagerCallback-377-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171249 INFO (jetty-launcher-374-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 11-ea+23
[junit4] 2> 171254 INFO (jetty-launcher-374-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 171254 INFO (jetty-launcher-374-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 171254 INFO (jetty-launcher-374-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 171256 INFO (jetty-launcher-374-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@17835305{/solr,null,AVAILABLE}
[junit4] 2> 171257 INFO (jetty-launcher-374-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1b13a609{SSL,[ssl, http/1.1]}{127.0.0.1:43839}
[junit4] 2> 171257 INFO (jetty-launcher-374-thread-1) [ ] o.e.j.s.Server Started @171291ms
[junit4] 2> 171257 INFO (jetty-launcher-374-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43839}
[junit4] 2> 171257 ERROR (jetty-launcher-374-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 171257 INFO (jetty-launcher-374-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 171257 INFO (jetty-launcher-374-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.5.0
[junit4] 2> 171257 INFO (jetty-launcher-374-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 171257 INFO (jetty-launcher-374-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 171257 INFO (jetty-launcher-374-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-07-25T16:15:06.350494Z
[junit4] 2> 171258 INFO (zkConnectionManagerCallback-379-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171259 INFO (jetty-launcher-374-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 171326 INFO (jetty-launcher-374-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42939/solr
[junit4] 2> 171328 INFO (zkConnectionManagerCallback-383-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171329 INFO (zkConnectionManagerCallback-385-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171378 INFO (jetty-launcher-374-thread-1) [n:127.0.0.1:43839_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43839_solr
[junit4] 2> 171380 INFO (jetty-launcher-374-thread-1) [n:127.0.0.1:43839_solr ] o.a.s.c.Overseer Overseer (id=72188959841058819-127.0.0.1:43839_solr-n_0000000000) starting
[junit4] 2> 171385 INFO (zkConnectionManagerCallback-392-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171386 INFO (jetty-launcher-374-thread-1) [n:127.0.0.1:43839_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42939/solr ready
[junit4] 2> 171390 INFO (OverseerStateUpdate-72188959841058819-127.0.0.1:43839_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:43839_solr
[junit4] 2> 171390 INFO (jetty-launcher-374-thread-1) [n:127.0.0.1:43839_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43839_solr
[junit4] 2> 171393 INFO (zkCallback-384-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 171394 INFO (zkCallback-391-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 171402 INFO (jetty-launcher-374-thread-1) [n:127.0.0.1:43839_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 171415 INFO (jetty-launcher-374-thread-1) [n:127.0.0.1:43839_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43839.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@332e9051
[junit4] 2> 171420 INFO (jetty-launcher-374-thread-1) [n:127.0.0.1:43839_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43839.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@332e9051
[junit4] 2> 171420 INFO (jetty-launcher-374-thread-1) [n:127.0.0.1:43839_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43839.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@332e9051
[junit4] 2> 171421 INFO (jetty-launcher-374-thread-1) [n:127.0.0.1:43839_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster2-001/node1/.
[junit4] 2> 171431 INFO (zkConnectionManagerCallback-395-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171438 INFO (zkConnectionManagerCallback-398-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171439 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster1-001
[junit4] 2> 171439 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 171439 INFO (Thread-281) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 171439 INFO (Thread-281) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 171441 ERROR (Thread-281) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 171539 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.ZkTestServer start zk server on port:41441
[junit4] 2> 171541 INFO (zkConnectionManagerCallback-402-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171544 INFO (jetty-launcher-399-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 11-ea+23
[junit4] 2> 171545 INFO (jetty-launcher-399-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 171545 INFO (jetty-launcher-399-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 171545 INFO (jetty-launcher-399-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 171545 INFO (jetty-launcher-399-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a9eebf5{/solr,null,AVAILABLE}
[junit4] 2> 171546 INFO (jetty-launcher-399-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4df09fb4{SSL,[ssl, http/1.1]}{127.0.0.1:36353}
[junit4] 2> 171546 INFO (jetty-launcher-399-thread-1) [ ] o.e.j.s.Server Started @171580ms
[junit4] 2> 171546 INFO (jetty-launcher-399-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36353}
[junit4] 2> 171546 ERROR (jetty-launcher-399-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 171546 INFO (jetty-launcher-399-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 171546 INFO (jetty-launcher-399-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.5.0
[junit4] 2> 171546 INFO (jetty-launcher-399-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 171546 INFO (jetty-launcher-399-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 171546 INFO (jetty-launcher-399-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-07-25T16:15:06.639847Z
[junit4] 2> 171548 INFO (zkConnectionManagerCallback-404-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171548 INFO (jetty-launcher-399-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 171618 INFO (jetty-launcher-399-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41441/solr
[junit4] 2> 171619 INFO (zkConnectionManagerCallback-408-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171625 INFO (zkConnectionManagerCallback-410-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171666 INFO (jetty-launcher-399-thread-1) [n:127.0.0.1:36353_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36353_solr
[junit4] 2> 171667 INFO (jetty-launcher-399-thread-1) [n:127.0.0.1:36353_solr ] o.a.s.c.Overseer Overseer (id=72188959860326403-127.0.0.1:36353_solr-n_0000000000) starting
[junit4] 2> 171672 INFO (zkConnectionManagerCallback-417-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171674 INFO (jetty-launcher-399-thread-1) [n:127.0.0.1:36353_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41441/solr ready
[junit4] 2> 171675 INFO (OverseerStateUpdate-72188959860326403-127.0.0.1:36353_solr-n_0000000000) [ ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:36353_solr
[junit4] 2> 171676 INFO (jetty-launcher-399-thread-1) [n:127.0.0.1:36353_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36353_solr
[junit4] 2> 171677 INFO (zkCallback-409-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 171677 INFO (zkCallback-416-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 171687 INFO (jetty-launcher-399-thread-1) [n:127.0.0.1:36353_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 171698 INFO (jetty-launcher-399-thread-1) [n:127.0.0.1:36353_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36353.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@332e9051
[junit4] 2> 171704 INFO (jetty-launcher-399-thread-1) [n:127.0.0.1:36353_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36353.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@332e9051
[junit4] 2> 171704 INFO (jetty-launcher-399-thread-1) [n:127.0.0.1:36353_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36353.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@332e9051
[junit4] 2> 171705 INFO (jetty-launcher-399-thread-1) [n:127.0.0.1:36353_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster1-001/node1/.
[junit4] 2> 171720 INFO (zkConnectionManagerCallback-420-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171721 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1007779fd2e0005, likely client has closed socket
[junit4] 2> 171723 INFO (zkConnectionManagerCallback-423-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171724 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.c.CdcrBidirectionalTest cluster2 zkHost = 127.0.0.1:42939/solr
[junit4] 2> 171724 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.c.CdcrBidirectionalTest cluster1 zkHost = 127.0.0.1:41441/solr
[junit4] 2> 171725 INFO (zkConnectionManagerCallback-425-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171728 INFO (zkConnectionManagerCallback-429-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 171729 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 171729 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41441/solr ready
[junit4] 2> 171741 INFO (qtp1940679673-1332) [n:127.0.0.1:36353_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=cdcr-cluster1&maxShardsPerNode=2&name=cdcr-cluster1&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 171743 INFO (OverseerThreadFactory-517-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection cdcr-cluster1
[junit4] 2> 171847 INFO (OverseerStateUpdate-72188959860326403-127.0.0.1:36353_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-cluster1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:36353/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 171848 INFO (OverseerStateUpdate-72188959860326403-127.0.0.1:36353_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"cdcr-cluster1_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:36353/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 172069 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 172069 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 172078 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster1_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 173090 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 173095 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.IndexSchema [cdcr-cluster1_shard2_replica_n2] Schema name=minimal
[junit4] 2> 173095 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 173100 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 173100 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster1_shard2_replica_n2' using configuration from collection cdcr-cluster1, trusted=true
[junit4] 2> 173101 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36353.solr.core.cdcr-cluster1.shard2.replica_n2' (registry 'solr.core.cdcr-cluster1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@332e9051
[junit4] 2> 173104 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 173104 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore [[cdcr-cluster1_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster1-001/node1/cdcr-cluster1_shard2_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster1-001/node1/./cdcr-cluster1_shard2_replica_n2/data/]
[junit4] 2> 173108 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-cluster1_shard1_replica_n1] Schema name=minimal
[junit4] 2> 173110 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 173110 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster1_shard1_replica_n1' using configuration from collection cdcr-cluster1, trusted=true
[junit4] 2> 173110 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36353.solr.core.cdcr-cluster1.shard1.replica_n1' (registry 'solr.core.cdcr-cluster1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@332e9051
[junit4] 2> 173110 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 173110 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-cluster1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster1-001/node1/cdcr-cluster1_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster1-001/node1/./cdcr-cluster1_shard1_replica_n1/data/]
[junit4] 2> 173309 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 173309 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 173311 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 173311 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 173311 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 173312 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 173313 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 173313 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 173313 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@3e922510[cdcr-cluster1_shard2_replica_n2] main]
[junit4] 2> 173314 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster1
[junit4] 2> 173315 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@208d81ef[cdcr-cluster1_shard1_replica_n1] main]
[junit4] 2> 173315 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster1
[junit4] 2> 173315 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster1/managed-schema
[junit4] 2> 173315 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster1
[junit4] 2> 173315 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 173315 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster1
[junit4] 2> 173315 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster1/managed-schema
[junit4] 2> 173316 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 173316 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 173317 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 173324 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-cluster1/cdcr/state/buffer
[junit4] 2> 173327 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-cluster1/cdcr/state/process
[junit4] 2> 173350 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1606979743585927168
[junit4] 2> 173366 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1606979743602704384
[junit4] 2> 173370 INFO (searcherExecutor-522-thread-1-processing-n:127.0.0.1:36353_solr x:cdcr-cluster1_shard2_replica_n2 c:cdcr-cluster1 s:shard2 r:core_node4) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore [cdcr-cluster1_shard2_replica_n2] Registered new searcher Searcher@3e922510[cdcr-cluster1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 173371 INFO (searcherExecutor-523-thread-1-processing-n:127.0.0.1:36353_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-cluster1_shard1_replica_n1] Registered new searcher Searcher@208d81ef[cdcr-cluster1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 173377 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard1 to Terms{values={core_node3=0}, version=0}
[junit4] 2> 173379 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard2 to Terms{values={core_node4=0}, version=0}
[junit4] 2> 173397 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 173397 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 173397 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:36353/solr/cdcr-cluster1_shard1_replica_n1/
[junit4] 2> 173397 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 173397 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:36353/solr/cdcr-cluster1_shard1_replica_n1/ has no replicas
[junit4] 2> 173397 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 173399 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 173399 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 173399 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:36353/solr/cdcr-cluster1_shard2_replica_n2/
[junit4] 2> 173399 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 173399 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:36353/solr/cdcr-cluster1_shard2_replica_n2/ has no replicas
[junit4] 2> 173399 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 173401 INFO (zkCallback-409-thread-1) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster1:shard1
[junit4] 2> 173404 INFO (zkCallback-409-thread-1) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster1:shard2
[junit4] 2> 173404 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:36353/solr/cdcr-cluster1_shard1_replica_n1/ shard1
[junit4] 2> 173405 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:36353/solr/cdcr-cluster1_shard2_replica_n2/ shard2
[junit4] 2> 173508 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 173510 INFO (qtp1940679673-1327) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster1_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1432
[junit4] 2> 173512 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 173517 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1448
[junit4] 2> 173527 INFO (qtp1940679673-1332) [n:127.0.0.1:36353_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 173625 INFO (zkCallback-409-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/state.json] for collection [cdcr-cluster1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 173745 INFO (OverseerCollectionConfigSetProcessor-72188959860326403-127.0.0.1:36353_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 174527 INFO (qtp1940679673-1332) [n:127.0.0.1:36353_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=cdcr-cluster1&maxShardsPerNode=2&name=cdcr-cluster1&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2786
[junit4] 2> 174538 INFO (zkConnectionManagerCallback-433-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 174544 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1007779fc080007, likely client has closed socket
[junit4] 2> 174548 INFO (zkConnectionManagerCallback-437-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 174550 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 174551 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[61288D56A353D960]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42939/solr ready
[junit4] 2> 174585 INFO (qtp2139775375-1278) [n:127.0.0.1:43839_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=cdcr-cluster2&maxShardsPerNode=2&name=cdcr-cluster2&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 174590 INFO (OverseerThreadFactory-504-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection cdcr-cluster2
[junit4] 2> 174701 INFO (OverseerStateUpdate-72188959841058819-127.0.0.1:43839_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-cluster2_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43839/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 174705 INFO (OverseerStateUpdate-72188959841058819-127.0.0.1:43839_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"cdcr-cluster2_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43839/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 174930 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster2_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 174931 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 174981 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster2_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 175967 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 175972 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-cluster2_shard1_replica_n1] Schema name=minimal
[junit4] 2> 175973 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 175973 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster2_shard1_replica_n1' using configuration from collection cdcr-cluster2, trusted=true
[junit4] 2> 175973 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43839.solr.core.cdcr-cluster2.shard1.replica_n1' (registry 'solr.core.cdcr-cluster2.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@332e9051
[junit4] 2> 175973 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 175973 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-cluster2_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster2-001/node1/cdcr-cluster2_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster2-001/node1/./cdcr-cluster2_shard1_replica_n1/data/]
[junit4] 2> 175990 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 175995 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.IndexSchema [cdcr-cluster2_shard2_replica_n2] Schema name=minimal
[junit4] 2> 175996 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 175996 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster2_shard2_replica_n2' using configuration from collection cdcr-cluster2, trusted=true
[junit4] 2> 175996 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43839.solr.core.cdcr-cluster2.shard2.replica_n2' (registry 'solr.core.cdcr-cluster2.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@332e9051
[junit4] 2> 175996 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 175996 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore [[cdcr-cluster2_shard2_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster2-001/node1/cdcr-cluster2_shard2_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.cdcr.CdcrBidirectionalTest_61288D56A353D960-001/cdcr-cluster2-001/node1/./cdcr-cluster2_shard2_replica_n2/data/]
[junit4] 2> 176181 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 176181 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 176182 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 176182 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 176184 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 176184 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 176184 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@211d833b[cdcr-cluster2_shard1_replica_n1] main]
[junit4] 2> 176185 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster2
[junit4] 2> 176185 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 176185 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 176185 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster2
[junit4] 2> 176185 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster2/managed-schema
[junit4] 2> 176185 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 176185 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 176186 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-cluster2/cdcr/state/buffer
[junit4] 2> 176187 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@67c39bd7[cdcr-cluster2_shard2_replica_n2] main]
[junit4] 2> 176187 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-cluster2/cdcr/state/process
[junit4] 2> 176187 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster2
[junit4] 2> 176187 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster2
[junit4] 2> 176187 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster2/managed-schema
[junit4] 2> 176187 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 176188 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1606979746561785856
[junit4] 2> 176188 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 176188 INFO (searcherExecutor-534-thread-1-processing-n:127.0.0.1:43839_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-cluster2_shard1_replica_n1] Registered new searcher Searcher@211d833b[cdcr-cluster2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 176189 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1606979746562834432
[junit4] 2> 176190 INFO (searcherExecutor-535-thread-1-processing-n:127.0.0.1:43839_solr x:cdcr-cluster2_shard2_replica_n2 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SolrCore [cdcr-cluster2_shard2_replica_n2] Registered new searcher Searcher@67c39bd7[cdcr-cluster2_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 176192 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster2/terms/shard1 to Terms{values={core_node3=0}, version=0}
[junit4] 2> 176192 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster2/terms/shard2 to Terms{values={core_node4=0}, version=0}
[junit4] 2> 176193 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 176193 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 176193 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:43839/solr/cdcr-cluster2_shard1_replica_n1/
[junit4] 2> 176194 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 176194 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:43839/solr/cdcr-cluster2_shard1_replica_n1/ has no replicas
[junit4] 2> 176194 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 176194 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 176194 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 176194 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:43839/solr/cdcr-cluster2_shard2_replica_n2/
[junit4] 2> 176194 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 176194 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:43839/solr/cdcr-cluster2_shard2_replica_n2/ has no replicas
[junit4] 2> 176194 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 176195 INFO (zkCallback-384-thread-2) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster2:shard1
[junit4] 2> 176195 INFO (zkCallback-384-thread-2) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster2:shard2
[junit4] 2> 176196 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:43839/solr/cdcr-cluster2_shard1_replica_n1/ shard1
[junit4] 2> 176196 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:43839/solr/cdcr-cluster2_shard2_replica_n2/ shard2
[junit4] 2> 176298 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 176299 INFO (qtp2139775375-1407) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster2_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1333
[junit4] 2> 176347 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 176349 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster2_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1418
[junit4] 2> 176352 INFO (qtp2139775375-1278) [n:127.0.0.1:43839_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 176449 INFO (zkCallback-384-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster2/state.json] for collection [cdcr-cluster2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 176591 INFO (OverseerCollectionConfigSetProcessor-72188959841058819-127.0.0.1:43839_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 177352 INFO (qtp2139775375-1278) [n:127.0.0.1:43839_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=cdcr-cluster2&maxShardsPerNode=2&name=cdcr-cluster2&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2766
[junit4] 2> 177356 INFO (zkConnectionManagerCallback-444-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177357 INFO (qtp1940679673-1331) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 177358 INFO (qtp1940679673-1331) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42939/solr ready
[junit4] 2> 177376 INFO (qtp2139775375-1406) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 177376 INFO (qtp2139775375-1406) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n2] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 177379 INFO (qtp2139775375-1274) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 177379 INFO (qtp2139775375-1274) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 177380 INFO (qtp2139775375-1278) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=13
[junit4] 2> 177380 INFO (qtp1940679673-1331) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard1
[junit4] 2> 177380 INFO (qtp1940679673-1331) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2, shard: shard1
[junit4] 2> 177381 INFO (qtp1940679673-1331) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
[junit4] 2> 177382 INFO (cdcr-bootstrap-status-440-thread-1-processing-n:127.0.0.1:36353_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2 shard: shard1 leader: https://127.0.0.1:43839/solr/cdcr-cluster2_shard1_replica_n1/
[junit4] 2> 177385 INFO (zkCallback-409-thread-2) [ ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/cdcr/state/process @ cdcr-cluster1:shard1
[junit4] 2> 177385 INFO (zkCallback-409-thread-1) [ ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/cdcr/state/process @ cdcr-cluster1:shard2
[junit4] 2> 177385 INFO (qtp1940679673-1331) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-cluster1:4&action=start&wt=javabin&version=2} status=0 QTime=30
[junit4] 2> 177385 INFO (zkCallback-409-thread-2) [ ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-cluster1:shard1
[junit4] 2> 177385 INFO (zkCallback-409-thread-1) [ ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-cluster1:shard2
[junit4] 2> 177390 INFO (zkConnectionManagerCallback-449-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177390 INFO (qtp2139775375-1279) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=https://127.0.0.1:36353/solr/cdcr-cluster1_shard1_replica_n1/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 177391 INFO (zkCallback-409-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 177391 INFO (recoveryExecutor-381-thread-1-processing-n:127.0.0.1:43839_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 177391 INFO (zkCallback-409-thread-1) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42939/solr ready
[junit4] 2> 177392 INFO (qtp2139775375-1278) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 177392 INFO (cdcr-bootstrap-status-440-thread-1-processing-n:127.0.0.1:36353_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager CDCR bootstrap running for 1 seconds, sleeping for 2000 ms
[junit4] 2> 177401 INFO (qtp1940679673-1331) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1606979747833708544,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 177401 INFO (qtp1940679673-1331) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 177405 INFO (qtp1940679673-1331) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 177405 INFO (qtp1940679673-1331) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 177413 INFO (qtp1940679673-1333) [n:127.0.0.1:36353_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster1_shard1_replica_n1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 177413 INFO (qtp2139775375-1273) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 177414 INFO (qtp2139775375-1273) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 177414 INFO (recoveryExecutor-381-thread-1-processing-n:127.0.0.1:43839_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 177414 INFO (recoveryExecutor-381-thread-1-processing-n:127.0.0.1:43839_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 177414 INFO (recoveryExecutor-381-thread-1-processing-n:127.0.0.1:43839_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 177414 INFO (recoveryExecutor-381-thread-1-processing-n:127.0.0.1:43839_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 177414 INFO (recoveryExecutor-381-thread-1-processing-n:127.0.0.1:43839_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 177416 INFO (recoveryExecutor-381-thread-1-processing-n:127.0.0.1:43839_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node3) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@75f86188[cdcr-cluster2_shard1_replica_n1] main]
[junit4] 2> 177417 ERROR (qtp2139775375-1278) [n:127.0.0.1:43839_solr c:cdcr-cluster2 s:shard1 r:core_node3 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Error while requesting shard's checkpoints
[junit4] 2> at org.apache.solr.handler.CdcrRequestHandler.handleCollectionCheckpointAction(CdcrRequestHandler.java:428)
[junit4] 2> at org.apache.solr.handler.CdcrRequestHandler.handleRequestBody(CdcrRequestHandler.java:194)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
[junit4] 2> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2541)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:709)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.cal
[...truncated too long message...]
jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/api-all-1.0.0-M20.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/bcprov-jdk15on-1.54.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/byte-buddy-1.6.2.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/commons-collections-3.2.2.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/commons-math3-3.6.1.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/ehcache-core-2.4.4.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/jersey-core-1.9.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/jersey-server-1.9.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/jetty-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/jetty-util-6.1.26.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/mina-core-2.0.0-M5.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/mockito-core-2.6.2.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/netty-all-4.0.36.Final.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/test-lib/objenesis-2.5.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/build/analysis/icu/lucene-analyzers-icu-7.5.0-SNAPSHOT.jar:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/contrib/solr-analysis-extras/classes/java:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/build/queryparser/classes/test:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/build/backward-codecs/classes/test:/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/contrib/analysis-extras/lib/icu4j-62.1.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-launcher.jar:/home/jenkins/.ant/lib/ivy-2.4.0.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-netrexx.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-xalan2.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-javamail.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-net.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-regexp.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-oro.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jmf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-log4j.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jsch.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jai.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bsf.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit4.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-resolver.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-apache-bcel.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-junit.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-jdepend.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-testutil.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-antlr.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-swing.jar:/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2/lib/ant-commons-logging.jar:/home/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.6.0.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/temp/junit4-J1-20180725_161214_89514647383874997709328.events @/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/temp/junit4-J1-20180725_161214_89518023818257937862878.suites -stdin
[junit4] ERROR: JVM J1 ended with an exception: Forked process returned with error code: 134. Very likely a JVM crash. See process stdout at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/temp/junit4-J1-20180725_161214_895416949269314303426.sysout
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
[junit4] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[junit4] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[junit4] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[junit4] at java.base/java.lang.Thread.run(Thread.java:834)
BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:633: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:577: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/build.xml:59: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build.xml:267: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/common-build.xml:556: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:1568: The following error occurred while executing this line:
/home/jenkins/workspace/Lucene-Solr-7.x-Linux/lucene/common-build.xml:1092: At least one slave process threw an exception, first: Forked process returned with error code: 134. Very likely a JVM crash. See process stdout at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/temp/junit4-J1-20180725_161214_895416949269314303426.sysout
Total time: 71 minutes 11 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/var/lib/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2