You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2017/08/20 17:29:29 UTC
[JENKINS] Lucene-Solr-Tests-6.6 - Build # 33 - Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.6/33/
6 tests failed.
FAILED: org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
Error Message:
Document mismatch on target after sync expected:<2000> but was:<1100>
Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1100>
at __randomizedtesting.SeedInfo.seed([143B5CC779620DB:D506FE9590C09320]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
Stack Trace:
java.lang.AssertionError: Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
at __randomizedtesting.SeedInfo.seed([143B5CC779620DB:4936C17871A50F4E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkNoTwoShardsUseTheSameIndexDir(CollectionsAPIDistributedZkTest.java:646)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:524)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.hdfs.HdfsRecoveryZkTest
Error Message:
ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog] org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.update.HdfsTransactionLog at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42) at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130) at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:203) at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:153) at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:110) at org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:108) at sun.reflect.GeneratedConstructorAccessor225.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:760) at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:822) at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1088) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:947) at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830) at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:922) at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:564) at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
java.lang.AssertionError: ObjectTracker found 1 object(s) that were not released!!! [HdfsTransactionLog]
org.apache.solr.common.util.ObjectReleaseTracker$ObjectTrackerException: org.apache.solr.update.HdfsTransactionLog
at org.apache.solr.common.util.ObjectReleaseTracker.track(ObjectReleaseTracker.java:42)
at org.apache.solr.update.HdfsTransactionLog.<init>(HdfsTransactionLog.java:130)
at org.apache.solr.update.HdfsUpdateLog.init(HdfsUpdateLog.java:203)
at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:153)
at org.apache.solr.update.UpdateHandler.<init>(UpdateHandler.java:110)
at org.apache.solr.update.DirectUpdateHandler2.<init>(DirectUpdateHandler2.java:108)
at sun.reflect.GeneratedConstructorAccessor225.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:760)
at org.apache.solr.core.SolrCore.createUpdateHandler(SolrCore.java:822)
at org.apache.solr.core.SolrCore.initUpdateHandler(SolrCore.java:1088)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:947)
at org.apache.solr.core.SolrCore.<init>(SolrCore.java:830)
at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:922)
at org.apache.solr.core.CoreContainer.lambda$load$5(CoreContainer.java:564)
at com.codahale.metrics.InstrumentedExecutorService$InstrumentedCallable.call(InstrumentedExecutorService.java:197)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:229)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([143B5CC779620DB]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.SolrTestCaseJ4.teardownTestCases(SolrTestCaseJ4.java:308)
at sun.reflect.GeneratedMethodAccessor48.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:870)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.AutoCommitTest.testMaxTime
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([143B5CC779620DB:9BB7C82EE90CBCE7]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:901)
at org.apache.solr.update.AutoCommitTest.testMaxTime(AutoCommitTest.java:270)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>
request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:894)
... 40 more
FAILED: org.apache.solr.update.AutoCommitTest.testCommitWithin
Error Message:
Exception during query
Stack Trace:
java.lang.RuntimeException: Exception during query
at __randomizedtesting.SeedInfo.seed([143B5CC779620DB:BB91DAB4F4B8CECE]:0)
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:901)
at org.apache.solr.update.AutoCommitTest.testCommitWithin(AutoCommitTest.java:353)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: REQUEST FAILED: xpath=//result[@numFound=1]
xml response was: <?xml version="1.0" encoding="UTF-8"?>
<response>
<lst name="responseHeader"><int name="status">0</int><int name="QTime">0</int></lst><result name="response" numFound="0" start="0"></result>
</response>
request was:q=id:529&qt=standard&start=0&rows=20&version=2.2
at org.apache.solr.SolrTestCaseJ4.assertQ(SolrTestCaseJ4.java:894)
... 40 more
FAILED: org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure
Error Message:
Did not see a fully active cluster after 30 seconds
Stack Trace:
java.lang.AssertionError: Did not see a fully active cluster after 30 seconds
at __randomizedtesting.SeedInfo.seed([1D98932973FF238B:95AE317AAB50CB99]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure(TestCollectionStateWatchers.java:250)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11187 lines...]
[junit4] Suite: org.apache.solr.cloud.CdcrBootstrapTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_143B5CC779620DB-001/init-core-data-001
[junit4] 2> 418053 WARN (SUITE-CdcrBootstrapTest-seed#[143B5CC779620DB]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 418053 INFO (SUITE-CdcrBootstrapTest-seed#[143B5CC779620DB]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 418054 INFO (SUITE-CdcrBootstrapTest-seed#[143B5CC779620DB]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 418068 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testConvertClusterToCdcrAndBootstrap
[junit4] 2> 418068 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_143B5CC779620DB-001/cdcr-target-001
[junit4] 2> 418069 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 418080 INFO (Thread-344) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 418080 INFO (Thread-344) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 418180 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.ZkTestServer start zk server on port:39243
[junit4] 2> 418284 ERROR (Thread-344) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 418442 INFO (jetty-launcher-134-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 418507 INFO (jetty-launcher-134-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@44ab01a0{/solr,null,AVAILABLE}
[junit4] 2> 418508 INFO (jetty-launcher-134-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2c388ed7{HTTP/1.1,[http/1.1]}{127.0.0.1:35654}
[junit4] 2> 418508 INFO (jetty-launcher-134-thread-1) [ ] o.e.j.s.Server Started @432401ms
[junit4] 2> 418508 INFO (jetty-launcher-134-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35654}
[junit4] 2> 418508 ERROR (jetty-launcher-134-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 418508 INFO (jetty-launcher-134-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 418508 INFO (jetty-launcher-134-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 418508 INFO (jetty-launcher-134-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 418508 INFO (jetty-launcher-134-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-20T15:17:18.743Z
[junit4] 2> 418593 INFO (jetty-launcher-134-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 418808 INFO (jetty-launcher-134-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 418841 INFO (jetty-launcher-134-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39243/solr
[junit4] 2> 419152 INFO (jetty-launcher-134-thread-1) [n:127.0.0.1:35654_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 419153 INFO (jetty-launcher-134-thread-1) [n:127.0.0.1:35654_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35654_solr
[junit4] 2> 419231 INFO (jetty-launcher-134-thread-1) [n:127.0.0.1:35654_solr ] o.a.s.c.Overseer Overseer (id=98516483336634371-127.0.0.1:35654_solr-n_0000000000) starting
[junit4] 2> 419263 INFO (jetty-launcher-134-thread-1) [n:127.0.0.1:35654_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35654_solr
[junit4] 2> 419265 INFO (OverseerStateUpdate-98516483336634371-127.0.0.1:35654_solr-n_0000000000) [n:127.0.0.1:35654_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 419988 INFO (jetty-launcher-134-thread-1) [n:127.0.0.1:35654_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_143B5CC779620DB-001/cdcr-target-001/node1/.
[junit4] 1> Target zkHost = 127.0.0.1:39243/solr
[junit4] 2> 420605 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_143B5CC779620DB-001/cdcr-source-001
[junit4] 2> 420605 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 420608 INFO (Thread-354) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 420608 INFO (Thread-354) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 420666 ERROR (Thread-354) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 420710 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.ZkTestServer start zk server on port:33741
[junit4] 2> 420947 INFO (jetty-launcher-146-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 420948 INFO (jetty-launcher-146-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5221ab85{/solr,null,AVAILABLE}
[junit4] 2> 420948 INFO (jetty-launcher-146-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@58688132{HTTP/1.1,[http/1.1]}{127.0.0.1:42079}
[junit4] 2> 420948 INFO (jetty-launcher-146-thread-1) [ ] o.e.j.s.Server Started @434841ms
[junit4] 2> 420948 INFO (jetty-launcher-146-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42079}
[junit4] 2> 420949 ERROR (jetty-launcher-146-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 420998 INFO (jetty-launcher-146-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 420999 INFO (jetty-launcher-146-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 421173 INFO (jetty-launcher-146-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 421174 INFO (jetty-launcher-146-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-20T15:17:21.409Z
[junit4] 2> 421377 INFO (jetty-launcher-146-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 421442 INFO (jetty-launcher-146-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 421458 INFO (jetty-launcher-146-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33741/solr
[junit4] 2> 421749 INFO (jetty-launcher-146-thread-1) [n:127.0.0.1:42079_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 421750 INFO (jetty-launcher-146-thread-1) [n:127.0.0.1:42079_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42079_solr
[junit4] 2> 421750 INFO (jetty-launcher-146-thread-1) [n:127.0.0.1:42079_solr ] o.a.s.c.Overseer Overseer (id=98516483498049539-127.0.0.1:42079_solr-n_0000000000) starting
[junit4] 2> 421942 INFO (jetty-launcher-146-thread-1) [n:127.0.0.1:42079_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42079_solr
[junit4] 2> 421979 INFO (zkCallback-153-thread-1-processing-n:127.0.0.1:42079_solr) [n:127.0.0.1:42079_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 422711 INFO (jetty-launcher-146-thread-1) [n:127.0.0.1:42079_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_143B5CC779620DB-001/cdcr-source-001/node1/.
[junit4] 2> 423516 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 423517 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33741/solr ready
[junit4] 2> 423639 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 423746 INFO (OverseerThreadFactory-244-thread-1-processing-n:127.0.0.1:42079_solr) [n:127.0.0.1:42079_solr ] o.a.s.c.CreateCollectionCmd Create collection cdcr-source
[junit4] 2> 424007 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&name=cdcr-source_shard1_replica1&action=CREATE&numShards=1&collection=cdcr-source&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2
[junit4] 2> 424008 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 424244 INFO (zkCallback-153-thread-2-processing-n:127.0.0.1:42079_solr) [n:127.0.0.1:42079_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 425154 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 425202 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.IndexSchema [cdcr-source_shard1_replica1] Schema name=minimal
[junit4] 2> 425217 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 425217 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-source_shard1_replica1' using configuration from collection cdcr-source, trusted=true
[junit4] 2> 425217 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 425217 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [[cdcr-source_shard1_replica1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_143B5CC779620DB-001/cdcr-source-001/node1/cdcr-source_shard1_replica1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_143B5CC779620DB-001/cdcr-source-001/node1/./cdcr-source_shard1_replica1/data/]
[junit4] 2> 425614 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 425614 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 425615 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 425615 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 425663 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4a5d068c[cdcr-source_shard1_replica1] main]
[junit4] 2> 425685 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-source
[junit4] 2> 425686 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-source
[junit4] 2> 425687 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 425687 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576263741259907072
[junit4] 2> 425785 INFO (searcherExecutor-247-thread-1-processing-n:127.0.0.1:42079_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@4a5d068c[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 425961 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 425961 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 425962 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42079/solr/cdcr-source_shard1_replica1/
[junit4] 2> 425962 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 425962 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SyncStrategy http://127.0.0.1:42079/solr/cdcr-source_shard1_replica1/ has no replicas
[junit4] 2> 425962 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 426009 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42079/solr/cdcr-source_shard1_replica1/ shard1
[junit4] 2> 426204 INFO (zkCallback-153-thread-2-processing-n:127.0.0.1:42079_solr) [n:127.0.0.1:42079_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 426223 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 426239 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&name=cdcr-source_shard1_replica1&action=CREATE&numShards=1&collection=cdcr-source&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2231
[junit4] 2> 426272 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 426394 INFO (zkCallback-153-thread-2-processing-n:127.0.0.1:42079_solr) [n:127.0.0.1:42079_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 427279 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=3640
[junit4] 1> Adding 10 docs with commit=true, numDocs=100
[junit4] 2> 427381 INFO (qtp234443785-852) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2}{add=[source_0 (1576263743027806208), source_1 (1576263743030951936), source_2 (1576263743030951937), source_3 (1576263743030951938), source_4 (1576263743030951939), source_5 (1576263743030951940), source_6 (1576263743032000512), source_7 (1576263743032000513), source_8 (1576263743032000514), source_9 (1576263743032000515), ... (100 adds)]} 0 7
[junit4] 2> 427503 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 427503 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@9dba171
[junit4] 2> 427849 INFO (OverseerCollectionConfigSetProcessor-98516483498049539-127.0.0.1:42079_solr-n_0000000000) [n:127.0.0.1:42079_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 428316 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2697c027[cdcr-source_shard1_replica1] main]
[junit4] 2> 428317 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 428319 INFO (searcherExecutor-247-thread-1-processing-n:127.0.0.1:42079_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@2697c027[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100)))}
[junit4] 2> 428328 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 824
[junit4] 1> Adding 10 docs with commit=true, numDocs=200
[junit4] 2> 428482 INFO (qtp234443785-853) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2}{add=[source_100 (1576263744074285056), source_101 (1576263744092110848), source_102 (1576263744092110849), source_103 (1576263744092110850), source_104 (1576263744092110851), source_105 (1576263744092110852), source_106 (1576263744092110853), source_107 (1576263744092110854), source_108 (1576263744092110855), source_109 (1576263744092110856), ... (100 adds)]} 0 112
[junit4] 2> 428499 INFO (qtp234443785-855) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 428649 INFO (qtp234443785-855) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@9dba171
[junit4] 2> 429527 INFO (qtp234443785-855) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7d790a88[cdcr-source_shard1_replica1] main]
[junit4] 2> 429528 INFO (searcherExecutor-247-thread-1-processing-n:127.0.0.1:42079_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@7d790a88[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100)))}
[junit4] 2> 429529 INFO (qtp234443785-855) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 429529 INFO (qtp234443785-855) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 1030
[junit4] 1> Adding 10 docs with commit=true, numDocs=300
[junit4] 2> 429631 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2}{add=[source_200 (1576263745319993344), source_201 (1576263745323139072), source_202 (1576263745323139073), source_203 (1576263745323139074), source_204 (1576263745323139075), source_205 (1576263745323139076), source_206 (1576263745323139077), source_207 (1576263745323139078), source_208 (1576263745323139079), source_209 (1576263745323139080), ... (100 adds)]} 0 84
[junit4] 2> 429645 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 429645 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@9dba171
[junit4] 2> 430518 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@21a2ab0f[cdcr-source_shard1_replica1] main]
[junit4] 2> 430567 INFO (searcherExecutor-247-thread-1-processing-n:127.0.0.1:42079_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@21a2ab0f[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100)))}
[junit4] 2> 430568 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 430568 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 923
[junit4] 1> Adding 10 docs with commit=true, numDocs=400
[junit4] 2> 430628 INFO (qtp234443785-852) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2}{add=[source_300 (1576263746388492288), source_301 (1576263746425192448), source_302 (1576263746425192449), source_303 (1576263746425192450), source_304 (1576263746425192451), source_305 (1576263746425192452), source_306 (1576263746425192453), source_307 (1576263746425192454), source_308 (1576263746425192455), source_309 (1576263746425192456), ... (100 adds)]} 0 50
[junit4] 2> 430644 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 430644 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@9dba171
[junit4] 2> 431366 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@418229c3[cdcr-source_shard1_replica1] main]
[junit4] 2> 431413 INFO (searcherExecutor-247-thread-1-processing-n:127.0.0.1:42079_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@418229c3[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100)))}
[junit4] 2> 431414 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 431414 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 769
[junit4] 1> Adding 10 docs with commit=true, numDocs=500
[junit4] 2> 431478 INFO (qtp234443785-853) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2}{add=[source_400 (1576263747297607680), source_401 (1576263747299704832), source_402 (1576263747299704833), source_403 (1576263747299704834), source_404 (1576263747299704835), source_405 (1576263747299704836), source_406 (1576263747299704837), source_407 (1576263747299704838), source_408 (1576263747299704839), source_409 (1576263747299704840), ... (100 adds)]} 0 62
[junit4] 2> 431481 INFO (qtp234443785-853) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 431481 INFO (qtp234443785-853) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@9dba171
[junit4] 2> 432056 INFO (qtp234443785-853) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@19674338[cdcr-source_shard1_replica1] main]
[junit4] 2> 432056 INFO (searcherExecutor-247-thread-1-processing-n:127.0.0.1:42079_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@19674338[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100)))}
[junit4] 2> 432057 INFO (qtp234443785-853) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 432057 INFO (qtp234443785-853) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 575
[junit4] 1> Adding 10 docs with commit=true, numDocs=600
[junit4] 2> 432131 INFO (qtp234443785-855) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2}{add=[source_500 (1576263747972890624), source_501 (1576263747974987776), source_502 (1576263747974987777), source_503 (1576263747974987778), source_504 (1576263747974987779), source_505 (1576263747974987780), source_506 (1576263747974987781), source_507 (1576263747974987782), source_508 (1576263747974987783), source_509 (1576263747974987784), ... (100 adds)]} 0 41
[junit4] 2> 432177 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 432177 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@9dba171
[junit4] 2> 432692 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@28a1129b[cdcr-source_shard1_replica1] main]
[junit4] 2> 432705 INFO (searcherExecutor-247-thread-1-processing-n:127.0.0.1:42079_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@28a1129b[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100) Uninverting(_5(6.6.1):C100)))}
[junit4] 2> 432706 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 432706 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 529
[junit4] 1> Adding 10 docs with commit=true, numDocs=700
[junit4] 2> 432772 INFO (qtp234443785-852) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2}{add=[source_600 (1576263748621959168), source_601 (1576263748655513600), source_602 (1576263748655513601), source_603 (1576263748655513602), source_604 (1576263748655513603), source_605 (1576263748655513604), source_606 (1576263748656562176), source_607 (1576263748656562177), source_608 (1576263748656562178), source_609 (1576263748656562179), ... (100 adds)]} 0 63
[junit4] 2> 432774 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 432774 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@9dba171
[junit4] 2> 433501 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@32c175b3[cdcr-source_shard1_replica1] main]
[junit4] 2> 433547 INFO (searcherExecutor-247-thread-1-processing-n:127.0.0.1:42079_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@32c175b3[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100) Uninverting(_5(6.6.1):C100) Uninverting(_6(6.6.1):C100)))}
[junit4] 2> 433713 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 433713 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 939
[junit4] 1> Adding 10 docs with commit=true, numDocs=800
[junit4] 2> 433778 INFO (qtp234443785-853) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2}{add=[source_700 (1576263749710381056), source_701 (1576263749711429632), source_702 (1576263749711429633), source_703 (1576263749711429634), source_704 (1576263749711429635), source_705 (1576263749711429636), source_706 (1576263749711429637), source_707 (1576263749711429638), source_708 (1576263749711429639), source_709 (1576263749711429640), ... (100 adds)]} 0 32
[junit4] 2> 433780 INFO (qtp234443785-855) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 433780 INFO (qtp234443785-855) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@9dba171
[junit4] 2> 434910 INFO (qtp234443785-855) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@25cb933e[cdcr-source_shard1_replica1] main]
[junit4] 2> 434911 INFO (searcherExecutor-247-thread-1-processing-n:127.0.0.1:42079_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@25cb933e[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100) Uninverting(_5(6.6.1):C100) Uninverting(_6(6.6.1):C100) Uninverting(_7(6.6.1):C100)))}
[junit4] 2> 434912 INFO (qtp234443785-855) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 434912 INFO (qtp234443785-855) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 1132
[junit4] 1> Adding 10 docs with commit=true, numDocs=900
[junit4] 2> 434944 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2}{add=[source_800 (1576263750949797888), source_801 (1576263750950846464), source_802 (1576263750950846465), source_803 (1576263750950846466), source_804 (1576263750950846467), source_805 (1576263750950846468), source_806 (1576263750950846469), source_807 (1576263750950846470), source_808 (1576263750950846471), source_809 (1576263750950846472), ... (100 adds)]} 0 16
[junit4] 2> 434946 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 434962 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@9dba171
[junit4] 2> 435466 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@30ee8157[cdcr-source_shard1_replica1] main]
[junit4] 2> 435495 INFO (searcherExecutor-247-thread-1-processing-n:127.0.0.1:42079_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@30ee8157[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100) Uninverting(_5(6.6.1):C100) Uninverting(_6(6.6.1):C100) Uninverting(_7(6.6.1):C100) Uninverting(_8(6.6.1):C100)))}
[junit4] 2> 435496 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 435496 INFO (qtp234443785-851) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 549
[junit4] 1> Adding 10 docs with commit=true, numDocs=1000
[junit4] 2> 435577 INFO (qtp234443785-852) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2}{add=[source_900 (1576263751579992064), source_901 (1576263751581040640), source_902 (1576263751581040641), source_903 (1576263751581040642), source_904 (1576263751581040643), source_905 (1576263751581040644), source_906 (1576263751581040645), source_907 (1576263751581040646), source_908 (1576263751581040647), source_909 (1576263751581040648), ... (100 adds)]} 0 48
[junit4] 2> 435579 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 435580 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@9dba171
[junit4] 2> 435952 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@da947b8[cdcr-source_shard1_replica1] main]
[junit4] 2> 435986 INFO (searcherExecutor-247-thread-1-processing-n:127.0.0.1:42079_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@da947b8[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100) Uninverting(_5(6.6.1):C100) Uninverting(_6(6.6.1):C100) Uninverting(_7(6.6.1):C100) Uninverting(_8(6.6.1):C100) Uninverting(_9(6.6.1):C100)))}
[junit4] 2> 436116 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 436116 INFO (qtp234443785-848) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 536
[junit4] 2> 436123 INFO (qtp234443785-849) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/select params={q=*:*&_stateVer_=cdcr-source:3&wt=javabin&version=2} hits=1000 status=0 QTime=0
[junit4] 2> 436186 INFO (qtp234443785-853) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/get params={qt=/get&_stateVer_=cdcr-source:3&getVersions=1000&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 436558 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@58688132{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 436592 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=368818435
[junit4] 2> 436626 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 436626 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 436626 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 436868 INFO (zkCallback-153-thread-2-processing-n:127.0.0.1:42079_solr) [n:127.0.0.1:42079_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 436932 INFO (coreCloseExecutor-252-thread-1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@1c5bda08
[junit4] 2> 437748 INFO (coreCloseExecutor-252-thread-1) [n:127.0.0.1:42079_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.cdcr-source.shard1.replica1
[junit4] 2> 437791 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.Overseer Overseer (id=98516483498049539-127.0.0.1:42079_solr-n_0000000000) closing
[junit4] 2> 437792 INFO (OverseerStateUpdate-98516483498049539-127.0.0.1:42079_solr-n_0000000000) [n:127.0.0.1:42079_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42079_solr
[junit4] 2> 437823 WARN (zkCallback-153-thread-2-processing-n:127.0.0.1:42079_solr) [n:127.0.0.1:42079_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 437824 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5221ab85{/solr,null,UNAVAILABLE}
[junit4] 2> 437824 INFO (zkCallback-160-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 437825 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 437927 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1d02c099{/solr,null,AVAILABLE}
[junit4] 2> 437927 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@61ed2b63{HTTP/1.1,[http/1.1]}{127.0.0.1:42923}
[junit4] 2> 437927 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.e.j.s.Server Started @451820ms
[junit4] 2> 437962 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42923}
[junit4] 2> 437981 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 438000 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 438000 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 438000 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 438000 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-20T15:17:38.235Z
[junit4] 2> 438044 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 438111 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 438111 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33741/solr
[junit4] 2> 438419 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [n:127.0.0.1:42923_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 438419 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [n:127.0.0.1:42923_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42923_solr
[junit4] 2> 438420 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [n:127.0.0.1:42923_solr ] o.a.s.c.Overseer Overseer (id=98516483498049547-127.0.0.1:42923_solr-n_0000000001) starting
[junit4] 2> 438556 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [n:127.0.0.1:42923_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42923_solr
[junit4] 2> 438654 INFO (zkCallback-166-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 438752 INFO (zkCallback-160-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 439134 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [n:127.0.0.1:42923_solr ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_143B5CC779620DB-001/cdcr-source-001/node1/.
[junit4] 2> 439135 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [n:127.0.0.1:42923_solr ] o.a.s.c.CorePropertiesLocator Cores are: [cdcr-source_shard1_replica1]
[junit4] 2> 439340 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 439409 INFO (zkCallback-166-thread-2-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 440469 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 440532 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 440581 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.IndexSchema [cdcr-source_shard1_replica1] Schema name=minimal
[junit4] 2> 440595 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 440652 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/cdcr-source/managed-schema
[junit4] 2> 440654 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/cdcr-source/schema.xml to /configs/cdcr-source/schema.xml.bak
[junit4] 2> 440654 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-source_shard1_replica1' using configuration from collection cdcr-source, trusted=true
[junit4] 2> 440655 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 440655 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [[cdcr-source_shard1_replica1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_143B5CC779620DB-001/cdcr-source-001/node1/cdcr-source_shard1_replica1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_143B5CC779620DB-001/cdcr-source-001/node1/./cdcr-source_shard1_replica1/data/]
[junit4] 2> 441371 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 441371 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 441402 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 441402 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 441739 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@54f97192[cdcr-source_shard1_replica1] main]
[junit4] 2> 441754 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-source
[junit4] 2> 441754 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-source
[junit4] 2> 441756 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-source/managed-schema
[junit4] 2> 441772 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 441772 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 441790 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-source/cdcr/state/buffer
[junit4] 2> 441954 INFO (coreLoadExecutor-261-thread-1-processing-n:127.0.0.1:42923_solr) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-source/cdcr/state/process
[junit4] 2> 442170 WARN (cdcr-update-log-synchronizer-267-thread-1) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrUpdateLogSynchronizer Couldn't get last processed version from leader http://127.0.0.1:42079/solr/cdcr-source_shard1_replica1/: Server refused connection at: http://127.0.0.1:42079/solr/cdcr-source_shard1_replica1
[junit4] 2> 442282 INFO (searcherExecutor-262-thread-1-processing-n:127.0.0.1:42923_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@54f97192[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(6.6.1):C1000)))}
[junit4] 2> 442357 INFO (coreZkRegister-254-thread-1-processing-n:127.0.0.1:42923_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source) [n:127.0.0.1:42923_solr c:cdcr-source s:shard1 r:core_node2 x:cdcr-source_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 442429 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[143B5CC779620DB]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: cdcr-source failOnTimeout:true timeout (sec):330
[junit4] 1> -
[junit4] 1> replica:core_node1 rstate:down live:false
[junit4] 1> replica:core_node2 rstate:down live:true
[junit4] 2> 442465 INFO (zkC
[...truncated too long message...]
2_solr) [n:127.0.0.1:39672_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 478032 INFO (zkCallback-554-thread-4-processing-n:127.0.0.1:39672_solr) [n:127.0.0.1:39672_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/currentstate/state.json] for collection [currentstate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 478046 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e008984ed0017, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 478047 INFO (zkCallback-525-thread-3-processing-n:127.0.0.1:40617_solr) [n:127.0.0.1:40617_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 478054 INFO (coreCloseExecutor-1030-thread-2) [n:127.0.0.1:40617_solr c:falsepredicate s:shard4 r:core_node2 x:falsepredicate_shard4_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.falsepredicate.shard4.replica1
[junit4] 2> 478070 INFO (coreCloseExecutor-1030-thread-1) [n:127.0.0.1:40617_solr c:testcollection s:shard4 r:core_node4 x:testcollection_shard4_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testcollection.shard4.replica1
[junit4] 2> 478086 INFO (zkCallback-527-thread-4-processing-n:127.0.0.1:40904_solr) [n:127.0.0.1:40904_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 478180 INFO (coreCloseExecutor-1032-thread-3) [n:127.0.0.1:40904_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.delayed.shard1.replica1
[junit4] 2> 478213 INFO (zkCallback-525-thread-3-processing-n:127.0.0.1:40617_solr) [n:127.0.0.1:40617_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 478225 INFO (zkCallback-554-thread-5-processing-n:127.0.0.1:39672_solr) [n:127.0.0.1:39672_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 478229 INFO (coreCloseExecutor-1032-thread-4) [n:127.0.0.1:40904_solr c:waitforstate s:shard1 r:core_node1 x:waitforstate_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.waitforstate.shard1.replica1
[junit4] 2> 478241 INFO (zkCallback-527-thread-4-processing-n:127.0.0.1:40904_solr) [n:127.0.0.1:40904_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 478261 INFO (coreCloseExecutor-1032-thread-2) [n:127.0.0.1:40904_solr c:falsepredicate s:shard1 r:core_node1 x:falsepredicate_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.falsepredicate.shard1.replica1
[junit4] 2> 478261 INFO (jetty-closer-505-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6f9ef5b7{/solr,null,UNAVAILABLE}
[junit4] 2> 478290 INFO (coreCloseExecutor-1032-thread-1) [n:127.0.0.1:40904_solr c:testcollection s:shard1 r:core_node1 x:testcollection_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testcollection.shard1.replica1
[junit4] 2> 478306 INFO (jetty-closer-505-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@25ccdace{/solr,null,UNAVAILABLE}
[junit4] 2> 478326 INFO (zkCallback-525-thread-3-processing-n:127.0.0.1:40617_solr) [n:127.0.0.1:40617_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 478604 INFO (jetty-closer-505-thread-2) [ ] o.a.s.c.Overseer Overseer (id=98516832489373706-127.0.0.1:40617_solr-n_0000000001) closing
[junit4] 2> 478622 INFO (OverseerStateUpdate-98516832489373706-127.0.0.1:40617_solr-n_0000000001) [n:127.0.0.1:40617_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40617_solr
[junit4] 2> 478700 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e008984ed000a, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 479822 WARN (zkCallback-527-thread-4-processing-n:127.0.0.1:40904_solr) [n:127.0.0.1:40904_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 479823 INFO (jetty-closer-505-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@49c2b685{/solr,null,UNAVAILABLE}
[junit4] 2> 480190 WARN (zkCallback-525-thread-3-processing-n:127.0.0.1:40617_solr) [n:127.0.0.1:40617_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 480190 INFO (jetty-closer-505-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@52c5c909{/solr,null,UNAVAILABLE}
[junit4] 2> 480206 ERROR (SUITE-TestCollectionStateWatchers-seed#[1D98932973FF238B]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 480222 INFO (SUITE-TestCollectionStateWatchers-seed#[1D98932973FF238B]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41524 41524
[junit4] 2> 480341 INFO (Thread-522) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41524 41524
[junit4] 2> 480341 WARN (Thread-522) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 7 /solr/security.json
[junit4] 2> 7 /solr/configs/config
[junit4] 2> 3 /solr/collections/stateformat1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 18 /solr/collections/falsepredicate/state.json
[junit4] 2> 12 /solr/collections/waitforstate/state.json
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 6 /solr/collections/testcollection/state.json
[junit4] 2> 3 /solr/overseer_elect/election/98516832489373707-127.0.0.1:40904_solr-n_0000000002
[junit4] 2> 3 /solr/collections/stateformat1/state.json
[junit4] 2> 2 /solr/collections/currentstate/state.json
[junit4] 2> 2 /solr/collections/delayed/state.json
[junit4] 2> 2 /solr/collections/tobedeleted/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/collections
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> 480343 INFO (SUITE-TestCollectionStateWatchers-seed#[1D98932973FF238B]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J2/temp/solr.common.cloud.TestCollectionStateWatchers_1D98932973FF238B-001
[junit4] 2> Aug 20, 2017 4:47:17 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=460, maxMBSortInHeap=6.2086896066324435, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=zh-CN, timezone=America/Chihuahua
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=197491712,total=409993216
[junit4] 2> NOTE: All tests run in this JVM: [NoOpResponseParserTest, CompoundEvaluatorTest, StreamingTest, SolrExampleXMLTest, CloudSolrClientMultiConstructorTest, NaturalLogEvaluatorTest, TestSuggesterResponse, HttpSolrClientConPoolTest, TestJavaBinCodec, TestCoreAdmin, CubedRootEvaluatorTest, TangentEvaluatorTest, SchemaTest, HyperbolicSineEvaluatorTest, NamedListTest, AndEvaluatorTest, TestConfigSetAdminRequest, JettyWebappTest, SolrExceptionTest, LargeVolumeJettyTest, FacetFieldTest, SolrExampleBinaryTest, HttpSolrClientBuilderTest, LBHttpSolrClientBuilderTest, GraphExpressionTest, JDBCStreamTest, AddEvaluatorTest, ArcCosineEvaluatorTest, ArcTangentEvaluatorTest, CoalesceEvaluatorTest, CosineEvaluatorTest, EqualsEvaluatorTest, ExclusiveOrEvaluatorTest, FieldEvaluatorTest, GreaterThanEqualToEvaluatorTest, GreaterThanEvaluatorTest, HyperbolicCosineEvaluatorTest, HyperbolicTangentEvaluatorTest, LessThanEvaluatorTest, ModuloEvaluatorTest, MultiplyEvaluatorTest, OrEvaluatorTest, PowerEvaluatorTest, RawValueEvaluatorTest, SineEvaluatorTest, SquareRootEvaluatorTest, SubtractEvaluatorTest, StreamExpressionParserTest, ConcatOperationTest, TestDelegationTokenResponse, TestToleratedUpdateError, SolrZkClientTest, TestCollectionStateWatchers]
[junit4] Completed [129/130 (1!)] on J2 in 72.23s, 9 tests, 1 failure <<< FAILURES!
[...truncated 42666 lines...]
[JENKINS] Lucene-Solr-Tests-6.6 - Build # 38 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.6/38/
1 tests failed.
FAILED: org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
Error Message:
Document mismatch on target after sync expected:<2000> but was:<1100>
Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1100>
at __randomizedtesting.SeedInfo.seed([F4D092F781974211:2095D9AE66C1F1EA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12715 lines...]
[junit4] Suite: org.apache.solr.cloud.CdcrBootstrapTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_F4D092F781974211-001/init-core-data-001
[junit4] 2> 3485668 WARN (SUITE-CdcrBootstrapTest-seed#[F4D092F781974211]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 3485668 INFO (SUITE-CdcrBootstrapTest-seed#[F4D092F781974211]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields
[junit4] 2> 3485670 INFO (SUITE-CdcrBootstrapTest-seed#[F4D092F781974211]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 3485744 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[F4D092F781974211]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBootstrapWithContinousIndexingOnSourceCluster
[junit4] 2> 3485744 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[F4D092F781974211]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_F4D092F781974211-001/cdcr-target-001
[junit4] 2> 3485745 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[F4D092F781974211]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3485748 INFO (Thread-2878) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3485748 INFO (Thread-2878) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3485749 ERROR (Thread-2878) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3485849 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[F4D092F781974211]) [ ] o.a.s.c.ZkTestServer start zk server on port:35266
[junit4] 2> 3485910 INFO (jetty-launcher-1215-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3486004 INFO (jetty-launcher-1215-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@538857f9{/solr,null,AVAILABLE}
[junit4] 2> 3486006 INFO (jetty-launcher-1215-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@67f388e6{SSL,[ssl, http/1.1]}{127.0.0.1:37230}
[junit4] 2> 3486006 INFO (jetty-launcher-1215-thread-1) [ ] o.e.j.s.Server Started @3498951ms
[junit4] 2> 3486006 INFO (jetty-launcher-1215-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37230}
[junit4] 2> 3486006 ERROR (jetty-launcher-1215-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3486006 INFO (jetty-launcher-1215-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 3486006 INFO (jetty-launcher-1215-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3486006 INFO (jetty-launcher-1215-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3486006 INFO (jetty-launcher-1215-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-30T15:33:59.687Z
[junit4] 2> 3486021 INFO (jetty-launcher-1215-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3486062 INFO (jetty-launcher-1215-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3486062 INFO (jetty-launcher-1215-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35266/solr
[junit4] 2> 3486206 INFO (jetty-launcher-1215-thread-1) [n:127.0.0.1:37230_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3486265 INFO (jetty-launcher-1215-thread-1) [n:127.0.0.1:37230_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37230_solr
[junit4] 2> 3486266 INFO (jetty-launcher-1215-thread-1) [n:127.0.0.1:37230_solr ] o.a.s.c.Overseer Overseer (id=98573172043284483-127.0.0.1:37230_solr-n_0000000000) starting
[junit4] 2> 3486297 INFO (jetty-launcher-1215-thread-1) [n:127.0.0.1:37230_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37230_solr
[junit4] 2> 3486297 INFO (zkCallback-1222-thread-1-processing-n:127.0.0.1:37230_solr) [n:127.0.0.1:37230_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3486591 INFO (jetty-launcher-1215-thread-1) [n:127.0.0.1:37230_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_F4D092F781974211-001/cdcr-target-001/node1/.
[junit4] 1> Target zkHost = 127.0.0.1:35266/solr
[junit4] 2> 3486760 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[F4D092F781974211]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_F4D092F781974211-001/cdcr-source-001
[junit4] 2> 3486760 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[F4D092F781974211]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3486791 INFO (Thread-2888) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3486791 INFO (Thread-2888) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3486792 ERROR (Thread-2888) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3486892 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[F4D092F781974211]) [ ] o.a.s.c.ZkTestServer start zk server on port:44666
[junit4] 2> 3486900 INFO (jetty-launcher-1227-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 3486917 INFO (jetty-launcher-1227-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6257c10e{/solr,null,AVAILABLE}
[junit4] 2> 3486918 INFO (jetty-launcher-1227-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@39d872ef{SSL,[ssl, http/1.1]}{127.0.0.1:32911}
[junit4] 2> 3486918 INFO (jetty-launcher-1227-thread-1) [ ] o.e.j.s.Server Started @3499862ms
[junit4] 2> 3486918 INFO (jetty-launcher-1227-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=32911}
[junit4] 2> 3486918 ERROR (jetty-launcher-1227-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3486933 INFO (jetty-launcher-1227-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 3486933 INFO (jetty-launcher-1227-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3486933 INFO (jetty-launcher-1227-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3486933 INFO (jetty-launcher-1227-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-30T15:34:00.614Z
[junit4] 2> 3486968 INFO (jetty-launcher-1227-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3487065 INFO (jetty-launcher-1227-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 3487066 INFO (jetty-launcher-1227-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44666/solr
[junit4] 2> 3487147 INFO (jetty-launcher-1227-thread-1) [n:127.0.0.1:32911_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3487148 INFO (jetty-launcher-1227-thread-1) [n:127.0.0.1:32911_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:32911_solr
[junit4] 2> 3487160 INFO (jetty-launcher-1227-thread-1) [n:127.0.0.1:32911_solr ] o.a.s.c.Overseer Overseer (id=98573172111638531-127.0.0.1:32911_solr-n_0000000000) starting
[junit4] 2> 3487199 INFO (jetty-launcher-1227-thread-1) [n:127.0.0.1:32911_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32911_solr
[junit4] 2> 3487248 INFO (OverseerStateUpdate-98573172111638531-127.0.0.1:32911_solr-n_0000000000) [n:127.0.0.1:32911_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3487409 INFO (jetty-launcher-1227-thread-1) [n:127.0.0.1:32911_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_F4D092F781974211-001/cdcr-source-001/node1/.
[junit4] 2> 3487879 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[F4D092F781974211]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3487898 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[F4D092F781974211]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44666/solr ready
[junit4] 2> 3488172 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3488188 INFO (OverseerThreadFactory-4119-thread-1-processing-n:127.0.0.1:32911_solr) [n:127.0.0.1:32911_solr ] o.a.s.c.CreateCollectionCmd Create collection cdcr-source
[junit4] 2> 3488334 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&name=cdcr-source_shard1_replica1&action=CREATE&numShards=1&collection=cdcr-source&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2
[junit4] 2> 3488335 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3488500 INFO (zkCallback-1234-thread-1-processing-n:127.0.0.1:32911_solr) [n:127.0.0.1:32911_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3489456 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 3489505 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 3489536 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.IndexSchema [cdcr-source_shard1_replica1] Schema name=minimal
[junit4] 2> 3489538 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3489538 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/cdcr-source/managed-schema
[junit4] 2> 3489569 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/cdcr-source/schema.xml to /configs/cdcr-source/schema.xml.bak
[junit4] 2> 3489569 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-source_shard1_replica1' using configuration from collection cdcr-source, trusted=true
[junit4] 2> 3489570 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3489570 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [[cdcr-source_shard1_replica1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_F4D092F781974211-001/cdcr-source-001/node1/cdcr-source_shard1_replica1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_F4D092F781974211-001/cdcr-source-001/node1/./cdcr-source_shard1_replica1/data/]
[junit4] 2> 3490116 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 3490116 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3490117 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3490117 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3490118 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2163e7a0[cdcr-source_shard1_replica1] main]
[junit4] 2> 3490142 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-source
[junit4] 2> 3490143 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-source
[junit4] 2> 3490143 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-source/managed-schema
[junit4] 2> 3490205 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 3490205 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3490238 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-source/cdcr/state/buffer
[junit4] 2> 3490239 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-source/cdcr/state/process
[junit4] 2> 3490251 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577170757413240832
[junit4] 2> 3490257 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3490257 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3490257 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:32911/solr/cdcr-source_shard1_replica1/
[junit4] 2> 3490257 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3490257 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:32911/solr/cdcr-source_shard1_replica1/ has no replicas
[junit4] 2> 3490257 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3490261 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@2163e7a0[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3490379 INFO (zkCallback-1234-thread-1-processing-n:127.0.0.1:32911_solr) [n:127.0.0.1:32911_solr ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-source:shard1
[junit4] 2> 3490424 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:32911/solr/cdcr-source_shard1_replica1/ shard1
[junit4] 2> 3490534 INFO (zkCallback-1234-thread-1-processing-n:127.0.0.1:32911_solr) [n:127.0.0.1:32911_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3490578 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3490583 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 x:cdcr-source_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&name=cdcr-source_shard1_replica1&action=CREATE&numShards=1&collection=cdcr-source&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2248
[junit4] 2> 3490623 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 3490769 INFO (zkCallback-1234-thread-1-processing-n:127.0.0.1:32911_solr) [n:127.0.0.1:32911_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3491679 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=3507
[junit4] 1> Adding 10 docs with commit=true, numDocs=100
[junit4] 2> 3491762 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=63
[junit4] 2> 3491764 INFO (qtp220033639-9161) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3491764 INFO (qtp220033639-9161) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3492253 INFO (qtp220033639-9161) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@745e6d4f[cdcr-source_shard1_replica1] main]
[junit4] 2> 3492254 INFO (qtp220033639-9161) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3492265 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@745e6d4f[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100)))}
[junit4] 2> 3492265 INFO (qtp220033639-9161) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=501
[junit4] 1> Adding 10 docs with commit=true, numDocs=200
[junit4] 2> 3492290 INFO (OverseerCollectionConfigSetProcessor-98573172111638531-127.0.0.1:32911_solr-n_0000000000) [n:127.0.0.1:32911_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3492298 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=31
[junit4] 2> 3492300 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3492300 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3492903 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4c4cae38[cdcr-source_shard1_replica1] main]
[junit4] 2> 3492903 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3492903 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@4c4cae38[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100)))}
[junit4] 2> 3492903 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=603
[junit4] 1> Adding 10 docs with commit=true, numDocs=300
[junit4] 2> 3492951 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=45
[junit4] 2> 3492953 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3492953 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3493594 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@64fddca6[cdcr-source_shard1_replica1] main]
[junit4] 2> 3493595 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3493601 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@64fddca6[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100)))}
[junit4] 2> 3493604 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=651
[junit4] 1> Adding 10 docs with commit=true, numDocs=400
[junit4] 2> 3493689 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 3493719 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3493719 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3494069 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@c21c6f8[cdcr-source_shard1_replica1] main]
[junit4] 2> 3494070 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3494070 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@c21c6f8[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100)))}
[junit4] 2> 3494070 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=351
[junit4] 1> Adding 10 docs with commit=true, numDocs=500
[junit4] 2> 3494107 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3494114 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3494114 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3494778 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@71c39b20[cdcr-source_shard1_replica1] main]
[junit4] 2> 3494778 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3494778 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@71c39b20[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100)))}
[junit4] 2> 3494807 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=692
[junit4] 1> Adding 10 docs with commit=true, numDocs=600
[junit4] 2> 3494818 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 3494831 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3494831 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3495331 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5d4d3922[cdcr-source_shard1_replica1] main]
[junit4] 2> 3495331 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@5d4d3922[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100) Uninverting(_5(6.6.1):C100)))}
[junit4] 2> 3495331 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3495331 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=500
[junit4] 1> Adding 10 docs with commit=true, numDocs=700
[junit4] 2> 3495346 INFO (qtp220033639-9161) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3495359 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3495359 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3495800 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@434cd266[cdcr-source_shard1_replica1] main]
[junit4] 2> 3495800 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3495801 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@434cd266[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100) Uninverting(_5(6.6.1):C100) Uninverting(_6(6.6.1):C100)))}
[junit4] 2> 3495801 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=441
[junit4] 1> Adding 10 docs with commit=true, numDocs=800
[junit4] 2> 3495824 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3495846 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3495846 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3496055 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1cf0d53b[cdcr-source_shard1_replica1] main]
[junit4] 2> 3496056 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@1cf0d53b[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100) Uninverting(_5(6.6.1):C100) Uninverting(_6(6.6.1):C100) Uninverting(_7(6.6.1):C100)))}
[junit4] 2> 3496056 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3496056 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=210
[junit4] 1> Adding 10 docs with commit=true, numDocs=900
[junit4] 2> 3496085 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 3496099 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3496099 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3496517 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@459a2f27[cdcr-source_shard1_replica1] main]
[junit4] 2> 3496518 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@459a2f27[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100) Uninverting(_5(6.6.1):C100) Uninverting(_6(6.6.1):C100) Uninverting(_7(6.6.1):C100) Uninverting(_8(6.6.1):C100)))}
[junit4] 2> 3496518 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3496518 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=419
[junit4] 1> Adding 10 docs with commit=true, numDocs=1000
[junit4] 2> 3496551 INFO (qtp220033639-9161) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=30
[junit4] 2> 3496552 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3496553 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3496919 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5dc3ecec[cdcr-source_shard1_replica1] main]
[junit4] 2> 3496919 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3496920 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@5dc3ecec[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.6.1):C100) Uninverting(_1(6.6.1):C100) Uninverting(_2(6.6.1):C100) Uninverting(_3(6.6.1):C100) Uninverting(_4(6.6.1):C100) Uninverting(_5(6.6.1):C100) Uninverting(_6(6.6.1):C100) Uninverting(_7(6.6.1):C100) Uninverting(_8(6.6.1):C100) Uninverting(_9(6.6.1):C100)))}
[junit4] 2> 3496920 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=367
[junit4] 2> 3496922 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/select params={q=*:*&_stateVer_=cdcr-source:3&wt=javabin&version=2} hits=1000 status=0 QTime=0
[junit4] 2> 3497186 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[F4D092F781974211]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3497187 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[F4D092F781974211]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35266/solr ready
[junit4] 2> 3497464 INFO (qtp24912259-9122) [n:127.0.0.1:37230_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cdcr-target&name=cdcr-target&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3497480 INFO (OverseerThreadFactory-4111-thread-1-processing-n:127.0.0.1:37230_solr) [n:127.0.0.1:37230_solr ] o.a.s.c.CreateCollectionCmd Create collection cdcr-target
[junit4] 2> 3497696 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-target&newCollection=true&name=cdcr-target_shard1_replica1&action=CREATE&numShards=1&collection=cdcr-target&shard=shard1&wt=javabin&version=2
[junit4] 2> 3497697 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3497809 INFO (zkCallback-1222-thread-1-processing-n:127.0.0.1:37230_solr) [n:127.0.0.1:37230_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3498803 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 3498831 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 3498846 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.s.IndexSchema [cdcr-target_shard1_replica1] Schema name=minimal
[junit4] 2> 3498847 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 3498848 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/cdcr-target/managed-schema
[junit4] 2> 3498862 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/cdcr-target/schema.xml to /configs/cdcr-target/schema.xml.bak
[junit4] 2> 3498862 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-target_shard1_replica1' using configuration from collection cdcr-target, trusted=true
[junit4] 2> 3498862 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3498862 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.SolrCore [[cdcr-target_shard1_replica1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_F4D092F781974211-001/cdcr-target-001/node1/cdcr-target_shard1_replica1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_F4D092F781974211-001/cdcr-target-001/node1/./cdcr-target_shard1_replica1/data/]
[junit4] 2> 3499134 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 3499134 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3499135 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 3499135 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3499146 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1041bb87[cdcr-target_shard1_replica1] main]
[junit4] 2> 3499147 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-target
[junit4] 2> 3499148 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-target
[junit4] 2> 3499148 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-target/managed-schema
[junit4] 2> 3499148 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 3499148 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 3499150 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-target/cdcr/state/buffer
[junit4] 2> 3499163 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-target/cdcr/state/process
[junit4] 2> 3499164 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577170766759198720
[junit4] 2> 3499166 INFO (searcherExecutor-4128-thread-1-processing-n:127.0.0.1:37230_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.SolrCore [cdcr-target_shard1_replica1] Registered new searcher Searcher@1041bb87[cdcr-target_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3499181 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3499181 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3499181 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:37230/solr/cdcr-target_shard1_replica1/
[junit4] 2> 3499182 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 3499182 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:37230/solr/cdcr-target_shard1_replica1/ has no replicas
[junit4] 2> 3499182 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3499208 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:37230/solr/cdcr-target_shard1_replica1/ shard1
[junit4] 2> 3499209 INFO (zkCallback-1222-thread-1-processing-n:127.0.0.1:37230_solr) [n:127.0.0.1:37230_solr ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-target:shard1
[junit4] 2> 3499333 INFO (zkCallback-1222-thread-1-processing-n:127.0.0.1:37230_solr) [n:127.0.0.1:37230_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3499377 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3499396 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 x:cdcr-target_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-target&newCollection=true&name=cdcr-target_shard1_replica1&action=CREATE&numShards=1&collection=cdcr-target&shard=shard1&wt=javabin&version=2} status=0 QTime=1700
[junit4] 2> 3499412 INFO (qtp24912259-9122) [n:127.0.0.1:37230_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 3499482 INFO (OverseerCollectionConfigSetProcessor-98573172043284483-127.0.0.1:37230_solr-n_0000000000) [n:127.0.0.1:37230_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3499558 INFO (zkCallback-1222-thread-1-processing-n:127.0.0.1:37230_solr) [n:127.0.0.1:37230_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3500412 INFO (qtp24912259-9122) [n:127.0.0.1:37230_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=cdcr-target&name=cdcr-target&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2947
[junit4] 2> 3501439 INFO (qtp24912259-9123) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.c.S.Request [cdcr-target_shard1_replica1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-target:3&action=start&wt=javabin&version=2} status=0 QTime=17
[junit4] 2> 3501447 INFO (zkCallback-1222-thread-1-processing-n:127.0.0.1:37230_solr) [n:127.0.0.1:37230_solr ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/cdcr/state/process @ cdcr-target:shard1
[junit4] 2> 3501464 INFO (zkCallback-1222-thread-1-processing-n:127.0.0.1:37230_solr) [n:127.0.0.1:37230_solr ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-target:shard1
[junit4] 2> 3501501 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3501502 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35266/solr ready
[junit4] 2> 3501583 INFO (qtp24912259-9117) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 3501583 INFO (qtp24912259-9117) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.c.S.Request [cdcr-target_shard1_replica1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=29
[junit4] 2> 3501584 INFO (qtp24912259-9116) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.c.S.Request [cdcr-target_shard1_replica1] webapp=/solr path=/cdcr params={_stateVer_=cdcr-target:3&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=61
[junit4] 2> 3501584 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-target with checkpoint -1 @ cdcr-source:shard1
[junit4] 2> 3501584 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target, shard: shard1
[junit4] 2> 3501584 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
[junit4] 2> 3501598 INFO (cdcr-bootstrap-status-1246-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target shard: shard1 leader: https://127.0.0.1:37230/solr/cdcr-target_shard1_replica1/
[junit4] 2> 3501633 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-source:3&action=start&wt=javabin&version=2} status=0 QTime=169
[junit4] 1> Adding 10 docs with commit=true, numDocs=1100
[junit4] 2> 3501635 INFO (zkCallback-1234-thread-1-processing-n:127.0.0.1:32911_solr) [n:127.0.0.1:32911_solr ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/cdcr/state/process @ cdcr-source:shard1
[junit4] 2> 3501635 INFO (zkCallback-1234-thread-1-processing-n:127.0.0.1:32911_solr) [n:127.0.0.1:32911_solr ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-source:shard1
[junit4] 2> 3501638 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3501718 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3501718 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3501735 INFO (qtp24912259-9116) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.c.S.Request [cdcr-target_shard1_replica1] webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=https://127.0.0.1:32911/solr/cdcr-source_shard1_replica1/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=18
[junit4] 2> 3501748 INFO (qtp24912259-9123) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.c.S.Request [cdcr-target_shard1_replica1] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 3501749 WARN (cdcr-bootstrap-status-1246-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Bootstrap process was not found on target collection: cdcr-target shard: shard1, leader: https://127.0.0.1:37230/solr/cdcr-target_shard1_replica1/
[junit4] 2> 3501749 INFO (cdcr-bootstrap-status-1246-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target shard: shard1 leader: https://127.0.0.1:37230/solr/cdcr-target_shard1_replica1/
[junit4] 2> 3501760 INFO (recoveryExecutor-1220-thread-1-processing-n:127.0.0.1:37230_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 3501789 INFO (qtp24912259-9123) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.c.S.Request [cdcr-target_shard1_replica1] webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=https://127.0.0.1:32911/solr/cdcr-source_shard1_replica1/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=38
[junit4] 2> 3501799 INFO (qtp24912259-9120) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.c.S.Request [cdcr-target_shard1_replica1] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 3501799 INFO (cdcr-bootstrap-status-1246-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager CDCR bootstrap running for 1 seconds, sleeping for 2000 ms
[junit4] 2> 3503097 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c778ea1[cdcr-source_shard1_replica1] main]
[junit4] 2> 3503097 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3503098 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@5c778ea1[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(6.6.1):C1000) Uninverting(_b(6.6.1):C100)))}
[junit4] 2> 3503098 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3503098 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 3503098 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3503098 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2} status=0 QTime=1316
[junit4] 2> 3503231 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 3503231 INFO (recoveryExecutor-1220-thread-1-processing-n:127.0.0.1:37230_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's generation: 12
[junit4] 2> 3503231 INFO (recoveryExecutor-1220-thread-1-processing-n:127.0.0.1:37230_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's version: 1504107255399
[junit4] 2> 3503231 INFO (recoveryExecutor-1220-thread-1-processing-n:127.0.0.1:37230_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 3503232 INFO (recoveryExecutor-1220-thread-1-processing-n:127.0.0.1:37230_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 3503232 INFO (recoveryExecutor-1220-thread-1-processing-n:127.0.0.1:37230_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 3503244 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=1526
[junit4] 1> Adding 10 docs with commit=true, numDocs=1200
[junit4] 2> 3503293 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=46
[junit4] 2> 3503295 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3503295 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3503371 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4649, name=tlog.0000000000000000000.1577170758931578880}, {size=4770, name=tlog.0000000000000000001.1577170759527170048}, {size=4770, name=tlog.0000000000000000002.1577170760197210112}, {size=4770, name=tlog.0000000000000000003.1577170761016147968}, {size=4770, name=tlog.0000000000000000004.1577170761454452736}, {size=4770, name=tlog.0000000000000000005.1577170762201038848}, {size=4770, name=tlog.0000000000000000006.1577170762753638400}, {size=4770, name=tlog.0000000000000000007.1577170763255906304}, {size=4770, name=tlog.0000000000000000008.1577170763528536064}, {size=4770, name=tlog.0000000000000000009.1577170763986763776}, {size=4870, name=tlog.0000000000000000010.1577170769351278592}]
[junit4] 2> 3503371 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=138
[junit4] 2> 3503376 INFO (recoveryExecutor-1220-thread-1-processing-n:127.0.0.1:37230_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Number of files in latest index in master: 37
[junit4] 2> 3503376 INFO (recoveryExecutor-1220-thread-1-processing-n:127.0.0.1:37230_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Number of tlog files in master: 11
[junit4] 2> 3503426 INFO (recoveryExecutor-1220-thread-1-processing-n:127.0.0.1:37230_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 3503427 INFO (recoveryExecutor-1220-thread-1-processing-n:127.0.0.1:37230_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:37230_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@1cbb4587 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1176a774)
[junit4] 2> 3503437 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Lucene50_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503486 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Direct_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503522 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503530 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_LuceneFixedGap_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503566 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Lucene50_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503579 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Lucene50_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503582 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Direct_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503592 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Direct_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503604 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503606 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503608 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_LuceneFixedGap_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503610 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_LuceneFixedGap_0.tii&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503613 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2e10ffe[cdcr-source_shard1_replica1] main]
[junit4] 2> 3503613 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3503620 INFO (qtp220033639-9163) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Lucene50_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503622 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_LuceneFixedGap_0.tib&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503628 INFO (searcherExecutor-4122-thread-1-processing-n:127.0.0.1:32911_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.SolrCore [cdcr-source_shard1_replica1] Registered new searcher Searcher@2e10ffe[cdcr-source_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(6.6.1):C1000) Uninverting(_b(6.6.1):C100) Uninverting(_c(6.6.1):C100)))}
[junit4] 2> 3503628 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=332
[junit4] 1> Adding 10 docs with commit=true, numDocs=1300
[junit4] 2> 3503634 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503634 INFO (qtp220033639-9161) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/update params={_stateVer_=cdcr-source:3&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 3503635 INFO (qtp220033639-9161) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3503635 INFO (qtp220033639-9161) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@65b7e8cd
[junit4] 2> 3503652 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503654 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Direct_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503668 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503671 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_Direct_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503685 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_Direct_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503688 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_Direct_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503709 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503711 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503725 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503727 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503746 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_LuceneFixedGap_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503748 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_LuceneFixedGap_0.tii&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503790 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_Lucene50_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503792 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_Lucene50_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503795 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_LuceneFixedGap_0.tib&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503797 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_Lucene50_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503810 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_Lucene50_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503816 INFO (qtp220033639-9158) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503835 INFO (qtp220033639-9162) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503863 INFO (qtp220033639-9160) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_Direct_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503875 INFO (qtp220033639-9157) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_b_LuceneFixedGap_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503877 INFO (qtp220033639-9164) [n:127.0.0.1:32911_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=segments_c&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 3503878 INFO (recoveryExecutor-1220-thread-1-process
[...truncated too long message...]
[junit4] 2> 2 /solr/configs/cdcr-source
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/collections/cdcr-source/cdcr/state/process
[junit4] 2> 2 /solr/collections/cdcr-source/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/collections
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> 3684352 INFO (jetty-closer-1285-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7f77a60a{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 3684373 INFO (jetty-closer-1285-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1229779511
[junit4] 2> 3684373 INFO (jetty-closer-1285-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 3684373 INFO (jetty-closer-1285-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 3684373 INFO (jetty-closer-1285-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 3684420 INFO (coreCloseExecutor-4262-thread-1) [n:127.0.0.1:46470_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.c.SolrCore [cdcr-target_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@21e543fa
[junit4] 2> 3684420 INFO (coreCloseExecutor-4262-thread-1) [n:127.0.0.1:46470_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR handler @ cdcr-target:shard1
[junit4] 2> 3684421 INFO (coreCloseExecutor-4262-thread-1) [n:127.0.0.1:46470_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.cdcr-target.shard1.replica1
[junit4] 2> 3684453 INFO (zkCallback-1291-thread-2-processing-n:127.0.0.1:46470_solr) [n:127.0.0.1:46470_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3684487 INFO (jetty-closer-1285-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98573182872387587-127.0.0.1:46470_solr-n_0000000000) closing
[junit4] 2> 3684499 INFO (OverseerStateUpdate-98573182872387587-127.0.0.1:46470_solr-n_0000000000) [n:127.0.0.1:46470_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:46470_solr
[junit4] 2> 3686032 WARN (zkCallback-1291-thread-2-processing-n:127.0.0.1:46470_solr) [n:127.0.0.1:46470_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3686033 INFO (jetty-closer-1285-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3fb1de77{/solr,null,UNAVAILABLE}
[junit4] 2> 3686034 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[F4D092F781974211]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3686034 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[F4D092F781974211]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42060 42060
[junit4] 2> 3686344 INFO (Thread-2929) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42060 42060
[junit4] 2> 3686345 WARN (Thread-2929) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 2 /solr/configs/cdcr-target/managed-schema
[junit4] 2> 2 /solr/collections/cdcr-target/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/collections/cdcr-target/cdcr/state/process
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/collections/cdcr-target/cdcr/state/buffer
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 3686345 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[F4D092F781974211]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testConvertClusterToCdcrAndBootstrap
[junit4] 2> 3686346 INFO (SUITE-CdcrBootstrapTest-seed#[F4D092F781974211]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CdcrBootstrapTest_F4D092F781974211-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {_version_=Lucene50(blocksize=128), xyz=PostingsFormat(name=Direct), id=PostingsFormat(name=LuceneFixedGap)}, docValues:{}, maxPointsInLeafNode=1505, maxMBSortInHeap=5.3635271347436095, sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=sr-Latn-ME, timezone=America/Martinique
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=229661176,total=523239424
[junit4] 2> NOTE: All tests run in this JVM: [PeerSyncWithIndexFingerprintCachingTest, UpdateRequestProcessorFactoryTest, TestScoreJoinQPNoScore, SegmentsInfoRequestHandlerTest, TestConfigOverlay, DirectSolrSpellCheckerTest, TestSchemaManager, TestCharFilters, TestSmileRequest, GraphQueryTest, TestQuerySenderListener, HdfsDirectoryFactoryTest, DocValuesTest, JSONWriterTest, TestSolrDynamicMBean, ConvertedLegacyTest, TestSolrQueryParserDefaultOperatorResource, ForceLeaderTest, DocValuesMissingTest, HdfsCollectionsAPIDistributedZkTest, CopyFieldTest, TestTolerantUpdateProcessorCloud, PreAnalyzedFieldManagedSchemaCloudTest, TestCloudPivotFacet, OpenCloseCoreStressTest, TestTrieFacet, TestLMDirichletSimilarityFactory, AssignTest, SpellCheckComponentTest, TestTestInjection, TestCursorMarkWithoutUniqueKey, TestInitParams, SolrInfoMBeanTest, SpellCheckCollatorTest, RestartWhileUpdatingTest, TestObjectReleaseTracker, TestLeaderElectionZkExpiry, TestCustomSort, TestCloudManagedSchema, TestFieldCollectionResource, TestCSVResponseWriter, MoreLikeThisHandlerTest, LeaderFailoverAfterPartitionTest, CleanupOldIndexTest, TestFieldCacheWithThreads, DistributedMLTComponentTest, TermVectorComponentTest, TestConfigReload, SolrCoreMetricManagerTest, InfixSuggestersTest, TestSegmentSorting, AddSchemaFieldsUpdateProcessorFactoryTest, TestLRUCache, JsonLoaderTest, TestRandomRequestDistribution, BigEndianAscendingWordSerializerTest, TestDistributedStatsComponentCardinality, RAMDirectoryFactoryTest, TestFieldCache, HdfsRecoveryZkTest, CloudMLTQParserTest, SolrIndexConfigTest, BasicFunctionalityTest, TestElisionMultitermQuery, SharedFSAutoReplicaFailoverUtilsTest, TestExclusionRuleCollectionAccess, WordBreakSolrSpellCheckerTest, TestNoOpRegenerator, OutOfBoxZkACLAndCredentialsProvidersTest, TestStressVersions, TestConfigSetsAPIExclusivity, CollectionsAPISolrJTest, TestSolrCoreSnapshots, TestCollectionAPIs, MultiTermTest, HighlighterTest, TestCoreContainer, TestPerFieldSimilarityWithDefaultOverride, TestPartialUpdateDeduplication, SolrIndexSplitterTest, TestDeleteCollectionOnDownNodes, TestGraphTermsQParserPlugin, TermsComponentTest, EchoParamsTest, TestIntervalFaceting, UUIDUpdateProcessorFallbackTest, TestWordDelimiterFilterFactory, TestDFISimilarityFactory, SpellingQueryConverterTest, MoveReplicaTest, TestUpdate, TestChildDocTransformer, CoreAdminRequestStatusTest, ClusterStateUpdateTest, TestSearchPerf, IndexSchemaTest, CdcrRequestHandlerTest, SparseHLLTest, TestMaxScoreQueryParser, CollectionsAPIDistributedZkTest, CoreSorterTest, SpellCheckCollatorWithCollapseTest, TestSimpleTrackingShardHandler, SearchHandlerTest, TestNumericTerms64, TestSQLHandlerNonCloud, TestShardHandlerFactory, SoftAutoCommitTest, TestLRUStatsCache, AnalysisErrorHandlingTest, TestDistributedGrouping, TestGraphMLResponseWriter, HdfsWriteToMultipleCollectionsTest, URLClassifyProcessorTest, DistribCursorPagingTest, CSVRequestHandlerTest, TestTrie, TestLuceneMatchVersion, TestImpersonationWithHadoopAuth, TestRandomFaceting, TestZkChroot, ShardRoutingCustomTest, TestHashPartitioner, TestRangeQuery, CurrencyFieldXmlFileTest, SuggesterTSTTest, SpatialFilterTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, LukeRequestHandlerTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, LoggingHandlerTest, JSONWriterTest, TestPHPSerializedResponseWriter, TestSolrIndexConfig, TestSolrDeletionPolicy2, TestConfig, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, TestSweetSpotSimilarityFactory, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMJelinekMercerSimilarityFactory, TestIBSimilarityFactory, TestFastWriter, TimeZoneUtilsTest, TestFastOutputStream, DateMathParserTest, TestSystemIdResolver, TestSuggestSpellingConverter, DateFieldTest, ClusterStateTest, TestSolrJ, SliceStateTest, FileUtilsTest, CursorPagingTest, SolrTestCaseJ4Test, TestDistributedMissingSort, TestPathTrie, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, ConnectionReuseTest, BaseCdcrDistributedZkTest, CdcrBootstrapTest]
[junit4] Completed [496/713 (1!)] on J1 in 200.70s, 3 tests, 1 failure <<< FAILURES!
[...truncated 44179 lines...]
[JENKINS] Lucene-Solr-Tests-6.6 - Build # 37 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.6/37/
2 tests failed.
FAILED: org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([80AE51F04F5F24D4:79E3C25F732A695E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:280)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:992)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestSegmentSorting.testSegmentTerminateEarly
Error Message:
KeeperErrorCode = Session expired for /clusterstate.json
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
at __randomizedtesting.SeedInfo.seed([80AE51F04F5F24D4:5008965D930DB0BE]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1102)
at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:323)
at org.apache.solr.common.cloud.SolrZkClient$5.execute(SolrZkClient.java:320)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:320)
at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:374)
at org.apache.solr.cloud.MiniSolrCloudCluster.deleteAllCollections(MiniSolrCloudCluster.java:454)
at org.apache.solr.cloud.TestSegmentSorting.ensureClusterEmpty(TestSegmentSorting.java:62)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:965)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:47)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11256 lines...]
[junit4] Suite: org.apache.solr.cloud.ShardSplitTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/init-core-data-001
[junit4] 2> 402797 WARN (SUITE-ShardSplitTest-seed#[80AE51F04F5F24D4]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
[junit4] 2> 402797 INFO (SUITE-ShardSplitTest-seed#[80AE51F04F5F24D4]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 402799 INFO (SUITE-ShardSplitTest-seed#[80AE51F04F5F24D4]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 402799 INFO (SUITE-ShardSplitTest-seed#[80AE51F04F5F24D4]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 402894 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 402913 INFO (Thread-194) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 402913 INFO (Thread-194) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 402978 ERROR (Thread-194) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 403022 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.ZkTestServer start zk server on port:46117
[junit4] 2> 403205 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 403250 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 403251 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 403252 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 403253 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 403266 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 403267 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 403268 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 403269 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 403270 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 403299 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 403347 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e218557940001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 404220 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/control-001/cores/collection1
[junit4] 2> 404266 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 404526 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2ca8e43c{/,null,AVAILABLE}
[junit4] 2> 404528 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@35cdccef{HTTP/1.1,[http/1.1]}{127.0.0.1:35376}
[junit4] 2> 404528 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.Server Started @417917ms
[junit4] 2> 404528 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/tempDir-001/control/data, hostContext=/, hostPort=35376, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/control-001/cores}
[junit4] 2> 404529 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 404529 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 404529 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 404529 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 404529 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-27T02:29:02.028Z
[junit4] 2> 404627 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 404627 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/control-001/solr.xml
[junit4] 2> 404676 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 404697 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46117/solr
[junit4] 2> 404935 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:35376_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 404935 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:35376_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35376_
[junit4] 2> 404949 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:35376_ ] o.a.s.c.Overseer Overseer (id=98553098432413700-127.0.0.1:35376_-n_0000000000) starting
[junit4] 2> 405062 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:35376_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35376_
[junit4] 2> 405102 INFO (OverseerStateUpdate-98553098432413700-127.0.0.1:35376_-n_0000000000) [n:127.0.0.1:35376_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 405719 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:35376_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/control-001/cores
[junit4] 2> 405720 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:35376_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 405749 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 405798 INFO (OverseerStateUpdate-98553098432413700-127.0.0.1:35376_-n_0000000000) [n:127.0.0.1:35376_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 407017 WARN (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 407017 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 407085 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 407921 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 408119 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 408119 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 408119 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/control-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/control-001/cores/collection1/data/]
[junit4] 2> 408119 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bdfaa59
[junit4] 2> 408121 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1904024132903125]
[junit4] 2> 408199 WARN (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 408480 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 408480 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 408480 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 408480 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 408481 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 408547 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1813c0e3[collection1] main]
[junit4] 2> 408548 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 408561 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 408561 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 408562 INFO (coreLoadExecutor-555-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576849580816859136
[junit4] 2> 408628 INFO (searcherExecutor-556-thread-1-processing-n:127.0.0.1:35376_ x:collection1 c:control_collection) [n:127.0.0.1:35376_ c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1813c0e3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 408659 INFO (coreZkRegister-548-thread-1-processing-n:127.0.0.1:35376_ x:collection1 c:control_collection) [n:127.0.0.1:35376_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 408659 INFO (coreZkRegister-548-thread-1-processing-n:127.0.0.1:35376_ x:collection1 c:control_collection) [n:127.0.0.1:35376_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 408659 INFO (coreZkRegister-548-thread-1-processing-n:127.0.0.1:35376_ x:collection1 c:control_collection) [n:127.0.0.1:35376_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:35376/collection1/
[junit4] 2> 408659 INFO (coreZkRegister-548-thread-1-processing-n:127.0.0.1:35376_ x:collection1 c:control_collection) [n:127.0.0.1:35376_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 408659 INFO (coreZkRegister-548-thread-1-processing-n:127.0.0.1:35376_ x:collection1 c:control_collection) [n:127.0.0.1:35376_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:35376/collection1/ has no replicas
[junit4] 2> 408659 INFO (coreZkRegister-548-thread-1-processing-n:127.0.0.1:35376_ x:collection1 c:control_collection) [n:127.0.0.1:35376_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 408677 INFO (coreZkRegister-548-thread-1-processing-n:127.0.0.1:35376_ x:collection1 c:control_collection) [n:127.0.0.1:35376_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:35376/collection1/ shard1
[junit4] 2> 408828 INFO (coreZkRegister-548-thread-1-processing-n:127.0.0.1:35376_ x:collection1 c:control_collection) [n:127.0.0.1:35376_ c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 409098 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 409099 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46117/solr ready
[junit4] 2> 409099 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 410566 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-1-001/cores/collection1
[junit4] 2> 410567 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-1-001
[junit4] 2> 410567 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 410716 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4cf001cd{/,null,AVAILABLE}
[junit4] 2> 410716 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7195c3da{HTTP/1.1,[http/1.1]}{127.0.0.1:45208}
[junit4] 2> 410716 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.Server Started @424106ms
[junit4] 2> 410716 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=45208, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-1-001/cores}
[junit4] 2> 410717 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 410731 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 410731 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 410731 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 410731 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-27T02:29:08.230Z
[junit4] 2> 410943 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 410943 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-1-001/solr.xml
[junit4] 2> 411191 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 411206 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46117/solr
[junit4] 2> 411467 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:45208_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 411469 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:45208_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 411482 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:45208_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45208_
[junit4] 2> 411500 INFO (zkCallback-194-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 411531 INFO (zkCallback-198-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 411677 INFO (zkCallback-203-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 412343 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:45208_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-1-001/cores
[junit4] 2> 412343 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:45208_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 412365 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 412626 INFO (OverseerStateUpdate-98553098432413700-127.0.0.1:35376_-n_0000000000) [n:127.0.0.1:35376_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 413743 WARN (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 413744 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 413895 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 414737 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 414856 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 414856 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 414856 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-1-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 414856 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bdfaa59
[junit4] 2> 414858 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1904024132903125]
[junit4] 2> 414924 WARN (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 415172 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 415172 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 415172 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 415172 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 415188 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 415189 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5ad8a4b7[collection1] main]
[junit4] 2> 415237 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 415257 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 415257 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 415258 INFO (coreLoadExecutor-566-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576849587838124032
[junit4] 2> 415371 INFO (searcherExecutor-567-thread-1-processing-n:127.0.0.1:45208_ x:collection1 c:collection1) [n:127.0.0.1:45208_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5ad8a4b7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 415559 INFO (coreZkRegister-561-thread-1-processing-n:127.0.0.1:45208_ x:collection1 c:collection1) [n:127.0.0.1:45208_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 415559 INFO (coreZkRegister-561-thread-1-processing-n:127.0.0.1:45208_ x:collection1 c:collection1) [n:127.0.0.1:45208_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 415559 INFO (coreZkRegister-561-thread-1-processing-n:127.0.0.1:45208_ x:collection1 c:collection1) [n:127.0.0.1:45208_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45208/collection1/
[junit4] 2> 415559 INFO (coreZkRegister-561-thread-1-processing-n:127.0.0.1:45208_ x:collection1 c:collection1) [n:127.0.0.1:45208_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 415559 INFO (coreZkRegister-561-thread-1-processing-n:127.0.0.1:45208_ x:collection1 c:collection1) [n:127.0.0.1:45208_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:45208/collection1/ has no replicas
[junit4] 2> 415559 INFO (coreZkRegister-561-thread-1-processing-n:127.0.0.1:45208_ x:collection1 c:collection1) [n:127.0.0.1:45208_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 415592 INFO (coreZkRegister-561-thread-1-processing-n:127.0.0.1:45208_ x:collection1 c:collection1) [n:127.0.0.1:45208_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45208/collection1/ shard2
[junit4] 2> 415705 INFO (coreZkRegister-561-thread-1-processing-n:127.0.0.1:45208_ x:collection1 c:collection1) [n:127.0.0.1:45208_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 417019 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-2-001/cores/collection1
[junit4] 2> 417019 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-2-001
[junit4] 2> 417030 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 417175 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5dca59b5{/,null,AVAILABLE}
[junit4] 2> 417175 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4314801c{HTTP/1.1,[http/1.1]}{127.0.0.1:36651}
[junit4] 2> 417176 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.Server Started @430565ms
[junit4] 2> 417176 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=36651, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-2-001/cores}
[junit4] 2> 417176 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 417338 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 417338 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 417338 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 417338 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-27T02:29:14.837Z
[junit4] 2> 417406 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 417406 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-2-001/solr.xml
[junit4] 2> 417439 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 417440 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46117/solr
[junit4] 2> 417540 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:36651_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 417556 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:36651_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 417569 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:36651_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36651_
[junit4] 2> 417570 INFO (zkCallback-203-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 417571 INFO (zkCallback-198-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 417605 INFO (zkCallback-194-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 417670 INFO (zkCallback-209-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 418276 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:36651_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-2-001/cores
[junit4] 2> 418276 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:36651_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 418293 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 418294 INFO (OverseerStateUpdate-98553098432413700-127.0.0.1:35376_-n_0000000000) [n:127.0.0.1:35376_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 419468 WARN (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 419481 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 419653 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 420112 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 420309 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 420309 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 420309 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-2-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 420309 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bdfaa59
[junit4] 2> 420423 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1904024132903125]
[junit4] 2> 420463 WARN (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 420918 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 420918 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 420919 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 420919 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 420955 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 421024 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1c605936[collection1] main]
[junit4] 2> 421033 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 421033 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 421033 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 421034 INFO (coreLoadExecutor-577-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576849593894699008
[junit4] 2> 421117 INFO (searcherExecutor-578-thread-1-processing-n:127.0.0.1:36651_ x:collection1 c:collection1) [n:127.0.0.1:36651_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1c605936[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 421137 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:36651_ x:collection1 c:collection1) [n:127.0.0.1:36651_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 421137 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:36651_ x:collection1 c:collection1) [n:127.0.0.1:36651_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 421137 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:36651_ x:collection1 c:collection1) [n:127.0.0.1:36651_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36651/collection1/
[junit4] 2> 421137 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:36651_ x:collection1 c:collection1) [n:127.0.0.1:36651_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 421137 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:36651_ x:collection1 c:collection1) [n:127.0.0.1:36651_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy http://127.0.0.1:36651/collection1/ has no replicas
[junit4] 2> 421137 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:36651_ x:collection1 c:collection1) [n:127.0.0.1:36651_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 421184 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:36651_ x:collection1 c:collection1) [n:127.0.0.1:36651_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36651/collection1/ shard1
[junit4] 2> 421361 INFO (coreZkRegister-572-thread-1-processing-n:127.0.0.1:36651_ x:collection1 c:collection1) [n:127.0.0.1:36651_ c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 422489 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-3-001/cores/collection1
[junit4] 2> 422489 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-3-001
[junit4] 2> 422518 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 422604 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@78d92041{/,null,AVAILABLE}
[junit4] 2> 422604 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@785a6b76{HTTP/1.1,[http/1.1]}{127.0.0.1:39184}
[junit4] 2> 422604 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.Server Started @435993ms
[junit4] 2> 422604 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/, hostPort=39184, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-3-001/cores}
[junit4] 2> 422604 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 422686 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 422686 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 422686 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 422686 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-27T02:29:20.185Z
[junit4] 2> 422812 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 422812 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-3-001/solr.xml
[junit4] 2> 422920 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 422969 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46117/solr
[junit4] 2> 423326 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:39184_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 423328 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:39184_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 423351 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:39184_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39184_
[junit4] 2> 423352 INFO (zkCallback-209-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 423352 INFO (zkCallback-194-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 423352 INFO (zkCallback-203-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 423352 INFO (zkCallback-198-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 423541 INFO (zkCallback-215-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 424149 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:39184_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-3-001/cores
[junit4] 2> 424149 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:39184_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 424163 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 424167 INFO (OverseerStateUpdate-98553098432413700-127.0.0.1:35376_-n_0000000000) [n:127.0.0.1:35376_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 425372 WARN (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 425373 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 425509 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 426238 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 426303 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 426303 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 426303 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-3-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 426303 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bdfaa59
[junit4] 2> 426334 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1904024132903125]
[junit4] 2> 426532 WARN (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 426792 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 426792 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 426793 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 426793 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 426794 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 426795 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@442b3401[collection1] main]
[junit4] 2> 426795 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 426796 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 426796 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 426825 INFO (searcherExecutor-589-thread-1-processing-n:127.0.0.1:39184_ x:collection1 c:collection1) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@442b3401[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 426845 INFO (coreLoadExecutor-588-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576849599987974144
[junit4] 2> 426946 INFO (coreZkRegister-583-thread-1-processing-n:127.0.0.1:39184_ x:collection1 c:collection1) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 426991 INFO (updateExecutor-212-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 427007 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 427008 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 427008 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 427008 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 427008 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [http://127.0.0.1:45208/collection1/] and I am [http://127.0.0.1:39184/collection1/]
[junit4] 2> 427056 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:45208]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:39184_&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 427106 INFO (qtp1294586579-958) [n:127.0.0.1:45208_ ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 427107 INFO (qtp1294586579-958) [n:127.0.0.1:45208_ ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard2 of collection1) have state: recovering
[junit4] 2> 427107 INFO (qtp1294586579-958) [n:127.0.0.1:45208_ ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:39184_, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"http://127.0.0.1:39184","node_name":"127.0.0.1:39184_","state":"recovering"}
[junit4] 2> 427107 INFO (qtp1294586579-958) [n:127.0.0.1:45208_ ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
[junit4] 2> 427107 INFO (qtp1294586579-958) [n:127.0.0.1:45208_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:39184_&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 427612 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [http://127.0.0.1:45208/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 427612 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=http://127.0.0.1:39184 START replicas=[http://127.0.0.1:45208/collection1/] nUpdates=100
[junit4] 2> 427631 INFO (qtp1294586579-955) [n:127.0.0.1:45208_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:0.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 427632 INFO (qtp1294586579-955) [n:127.0.0.1:45208_ c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp= path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 427726 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:93.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 427726 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 427726 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 427726 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 427727 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 427727 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 427727 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 427727 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 427727 INFO (recoveryExecutor-213-thread-1-processing-n:127.0.0.1:39184_ x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:39184_ c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 428041 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-4-001/cores/collection1
[junit4] 2> 428042 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-4-001
[junit4] 2> 428056 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 428153 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@27df610c{/,null,AVAILABLE}
[junit4] 2> 428153 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@50ae98c{HTTP/1.1,[http/1.1]}{127.0.0.1:34956}
[junit4] 2> 428154 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.e.j.s.Server Started @441542ms
[junit4] 2> 428154 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/tempDir-001/jetty4, solrconfig=solrconfig.xml, hostContext=/, hostPort=34956, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-4-001/cores}
[junit4] 2> 428154 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 428154 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 428154 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 428154 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 428154 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-27T02:29:25.653Z
[junit4] 2> 428353 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 428353 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-4-001/solr.xml
[junit4] 2> 428384 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 428385 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46117/solr
[junit4] 2> 428919 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:34956_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 428933 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:34956_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 428934 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:34956_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34956_
[junit4] 2> 428935 INFO (zkCallback-203-thread-1-processing-n:127.0.0.1:45208_) [n:127.0.0.1:45208_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 428941 INFO (zkCallback-215-thread-1-processing-n:127.0.0.1:39184_) [n:127.0.0.1:39184_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 429140 INFO (zkCallback-198-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 429140 INFO (zkCallback-209-thread-1-processing-n:127.0.0.1:36651_) [n:127.0.0.1:36651_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 429141 INFO (zkCallback-194-thread-1-processing-n:127.0.0.1:35376_) [n:127.0.0.1:35376_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 429257 INFO (zkCallback-222-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 429505 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:34956_ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-4-001/cores
[junit4] 2> 429505 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[80AE51F04F5F24D4]) [n:127.0.0.1:34956_ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 429558 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 429571 INFO (OverseerStateUpdate-98553098432413700-127.0.0.1:35376_-n_0000000000) [n:127.0.0.1:35376_ ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 430673 WARN (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.c.Config Beginning with Solr 5.5, <mergePolicy> is deprecated, use <mergePolicyFactory> instead.
[junit4] 2> 430674 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 430745 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 431308 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 431357 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 431357 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 431357 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-4-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J2/temp/solr.cloud.ShardSplitTest_80AE51F04F5F24D4-001/shard-4-001/cores/collection1/data/]
[junit4] 2> 431357 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@4bdfaa59
[junit4] 2> 431371 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1904024132903125]
[junit4] 2> 431489 WARN (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 431761 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 431761 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 431762 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 431762 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 431763 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=38, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 431814 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@55ade535[collection1] main]
[junit4] 2> 431853 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:34956_) [n:127.0.0.1:34956_ c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 431853 INFO (coreLoadExecutor-599-thread-1-processing-n:127.0.0.1:349
[...truncated too long message...]
rd1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@60aee177
[junit4] 2> 2297640 WARN (zkCallback-708-thread-8-processing-n:127.0.0.1:44043_solr) [n:127.0.0.1:44043_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2297640 INFO (zkCallback-709-thread-8-processing-n:127.0.0.1:39764_solr) [n:127.0.0.1:39764_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 2297641 INFO (zkCallback-706-thread-6-processing-n:127.0.0.1:44949_solr) [n:127.0.0.1:44949_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 2297641 INFO (zkCallback-693-thread-4-processing-n:127.0.0.1:36111_solr) [n:127.0.0.1:36111_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 2297641 INFO (coreCloseExecutor-2062-thread-1) [n:127.0.0.1:36111_solr c:testSegmentTerminateEarly s:shard2 r:core_node1 x:testSegmentTerminateEarly_shard2_replica1] o.a.s.c.SolrCore [testSegmentTerminateEarly_shard2_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@5b03c9c7
[junit4] 2> 2297664 INFO (zkCallback-699-thread-3-processing-n:127.0.0.1:36611_solr) [n:127.0.0.1:36611_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 2297688 INFO (coreCloseExecutor-2061-thread-1) [n:127.0.0.1:44949_solr c:testSegmentTerminateEarly s:shard2 r:core_node2 x:testSegmentTerminateEarly_shard2_replica2] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testSegmentTerminateEarly.shard2.replica2
[junit4] 2> 2297717 INFO (coreCloseExecutor-2062-thread-1) [n:127.0.0.1:36111_solr c:testSegmentTerminateEarly s:shard2 r:core_node1 x:testSegmentTerminateEarly_shard2_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testSegmentTerminateEarly.shard2.replica1
[junit4] 2> 2297720 INFO (jetty-closer-680-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@66a8bf1e{/solr,null,UNAVAILABLE}
[junit4] 2> 2297722 WARN (zkCallback-706-thread-6-processing-n:127.0.0.1:44949_solr) [n:127.0.0.1:44949_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2297722 INFO (zkCallback-699-thread-3-processing-n:127.0.0.1:36611_solr) [n:127.0.0.1:36611_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 2297722 INFO (zkCallback-693-thread-4-processing-n:127.0.0.1:36111_solr) [n:127.0.0.1:36111_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 2297722 INFO (zkCallback-709-thread-8-processing-n:127.0.0.1:39764_solr) [n:127.0.0.1:39764_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 2297763 INFO (coreCloseExecutor-2063-thread-1) [n:127.0.0.1:39764_solr c:testSegmentTerminateEarly s:shard1 r:core_node4 x:testSegmentTerminateEarly_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testSegmentTerminateEarly.shard1.replica1
[junit4] 2> 2297798 INFO (jetty-closer-680-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@29bb462b{/solr,null,UNAVAILABLE}
[junit4] 2> 2297798 INFO (zkCallback-693-thread-6-processing-n:127.0.0.1:36111_solr) [n:127.0.0.1:36111_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2297799 INFO (zkCallback-709-thread-6-processing-n:127.0.0.1:39764_solr) [n:127.0.0.1:39764_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2297806 INFO (jetty-closer-680-thread-2) [ ] o.a.s.c.Overseer Overseer (id=98553214690983944-127.0.0.1:36111_solr-n_0000000000) closing
[junit4] 2> 2297817 INFO (OverseerStateUpdate-98553214690983944-127.0.0.1:36111_solr-n_0000000000) [n:127.0.0.1:36111_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36111_solr
[junit4] 2> 2297977 WARN (zkCallback-693-thread-4-processing-n:127.0.0.1:36111_solr) [n:127.0.0.1:36111_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2297992 INFO (zkCallback-693-thread-6-processing-n:127.0.0.1:36111_solr) [n:127.0.0.1:36111_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 2298000 INFO (jetty-closer-680-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7b633b53{/solr,null,UNAVAILABLE}
[junit4] 2> 2299303 WARN (zkCallback-699-thread-3-processing-n:127.0.0.1:36611_solr) [n:127.0.0.1:36611_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2299303 INFO (jetty-closer-680-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6e43f5fb{/solr,null,UNAVAILABLE}
[junit4] 2> 2299420 WARN (zkCallback-709-thread-8-processing-n:127.0.0.1:39764_solr) [n:127.0.0.1:39764_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2299421 INFO (jetty-closer-680-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@249edf8{/solr,null,UNAVAILABLE}
[junit4] 2> 2299422 ERROR (SUITE-TestSegmentSorting-seed#[80AE51F04F5F24D4]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2299422 INFO (SUITE-TestSegmentSorting-seed#[80AE51F04F5F24D4]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:32869 32869
[junit4] 2> 2299533 INFO (Thread-2419) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:32869 32869
[junit4] 2> 2299542 WARN (Thread-2419) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/configs/class org.apache.solr.cloud.TestSegmentSorting_configSet
[junit4] 2> 7 /solr/aliases.json
[junit4] 2> 7 /solr/clusterprops.json
[junit4] 2> 5 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/testSegmentTerminateEarly/state.json
[junit4] 2> 4 /solr/collections/testAtomicUpdateOfSegmentSortField/state.json
[junit4] 2> 2 /solr/overseer_elect/election/98553214690983949-127.0.0.1:44949_solr-n_0000000002
[junit4] 2> 2 /solr/overseer_elect/election/98553214690983945-127.0.0.1:36611_solr-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/98553214690983944-127.0.0.1:36111_solr-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 9 /solr/collections
[junit4] 2> 7 /solr/live_nodes
[junit4] 2>
[junit4] 2> 2299542 INFO (SUITE-TestSegmentSorting-seed#[80AE51F04F5F24D4]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.TestSegmentSorting_80AE51F04F5F24D4-001
[junit4] 2> Aug 27, 2017 3:00:36 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true,coord=yes): {}, locale=is-IS, timezone=Europe/Belfast
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=129453056,total=513802240
[junit4] 2> NOTE: All tests run in this JVM: [AddSchemaFieldsUpdateProcessorFactoryTest, TestShardHandlerFactory, TestDistributedGrouping, SpellCheckCollatorTest, TestSolrIndexConfig, TestTestInjection, ForceLeaderTest, HdfsRecoveryZkTest, DocValuesTest, TestCustomDocTransformer, HdfsDirectoryFactoryTest, TestIntervalFaceting, TestStressCloudBlindAtomicUpdates, ChaosMonkeySafeLeaderTest, ShardRoutingCustomTest, DistributedDebugComponentTest, HdfsWriteToMultipleCollectionsTest, DistributedIntervalFacetingTest, DistanceFunctionTest, TestElisionMultitermQuery, DocValuesMissingTest, TestPartialUpdateDeduplication, SegmentsInfoRequestHandlerTest, TestEmbeddedSolrServerConstructors, TestNoOpRegenerator, TestRawResponseWriter, HdfsDirectoryTest, TestChildDocTransformer, TestSQLHandlerNonCloud, TestFieldCache, SimplePostToolTest, FullHLLTest, TestPerFieldSimilarityWithDefaultOverride, TestSearchPerf, HdfsThreadLeakTest, BlockCacheTest, MultiTermTest, DocumentBuilderTest, TestSurroundQueryParser, TestInPlaceUpdatesStandalone, TestRandomRequestDistribution, TestCSVResponseWriter, TestConfigSets, AnalysisErrorHandlingTest, DistribCursorPagingTest, TestFieldCacheSort, PreAnalyzedFieldManagedSchemaCloudTest, IndexBasedSpellCheckerTest, HighlighterTest, CursorMarkTest, LoggingHandlerTest, RemoteQueryErrorTest, TestTrieFacet, TermsComponentTest, TestDFISimilarityFactory, TestCloudPivotFacet, TestStressVersions, MoveReplicaTest, DistributedQueryComponentOptimizationTest, SolrMetricsIntegrationTest, SpellCheckComponentTest, PKIAuthenticationIntegrationTest, OverseerModifyCollectionTest, JsonLoaderTest, TestImpersonationWithHadoopAuth, TestTrie, SearchHandlerTest, TermVectorComponentTest, CollectionsAPISolrJTest, TestObjectReleaseTracker, TestSegmentSorting]
[junit4] Completed [199/713 (2!)] on J1 in 123.04s, 2 tests, 1 error <<< FAILURES!
[...truncated 45217 lines...]
[JENKINS] Lucene-Solr-Tests-6.6 - Build # 36 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.6/36/
2 tests failed.
FAILED: org.apache.solr.handler.TestSolrConfigHandlerConcurrent.test
Error Message:
Could not load collection from ZK: collection1
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: collection1
at __randomizedtesting.SeedInfo.seed([392477BBD94A556C:B170486177B63894]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1098)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:638)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:213)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.getTotalReplicas(AbstractFullDistribZkTestBase.java:450)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:408)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:331)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:990)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:967)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:357)
at org.apache.solr.common.cloud.SolrZkClient$7.execute(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:354)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1110)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1096)
... 38 more
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
Stack Trace:
java.lang.AssertionError: Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
at __randomizedtesting.SeedInfo.seed([392477BBD94A556C:7151030FDF797AF9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkNoTwoShardsUseTheSameIndexDir(CollectionsAPIDistributedZkTest.java:646)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:524)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11221 lines...]
[junit4] Suite: org.apache.solr.handler.TestSolrConfigHandlerConcurrent
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/init-core-data-001
[junit4] 2> 791570 WARN (SUITE-TestSolrConfigHandlerConcurrent-seed#[392477BBD94A556C]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 791570 INFO (SUITE-TestSolrConfigHandlerConcurrent-seed#[392477BBD94A556C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 791571 INFO (SUITE-TestSolrConfigHandlerConcurrent-seed#[392477BBD94A556C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 791572 INFO (SUITE-TestSolrConfigHandlerConcurrent-seed#[392477BBD94A556C]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /u_v/
[junit4] 2> 791780 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 791808 INFO (Thread-167) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 791808 INFO (Thread-167) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 791832 ERROR (Thread-167) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 791892 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.ZkTestServer start zk server on port:46684
[junit4] 2> 792417 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 792449 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 792483 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 792484 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 792485 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 792486 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 792517 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 792518 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 792518 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 792548 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 792549 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 792564 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e115fa1de0001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 794086 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/control-001/cores/collection1
[junit4] 2> 794136 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 794279 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@e236d40{/u_v,null,AVAILABLE}
[junit4] 2> 794282 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@506cf608{SSL,[ssl, http/1.1]}{127.0.0.1:40523}
[junit4] 2> 794282 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.Server Started @806668ms
[junit4] 2> 794282 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/tempDir-001/control/data, hostContext=/u_v, hostPort=40523, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/control-001/cores}
[junit4] 2> 794282 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 794282 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 794283 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 794283 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 794283 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-23T23:13:56.100Z
[junit4] 2> 794591 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 794592 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/control-001/solr.xml
[junit4] 2> 794628 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 794645 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46684/solr
[junit4] 2> 795177 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:40523_u_v ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 795178 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:40523_u_v ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40523_u_v
[junit4] 2> 795179 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:40523_u_v ] o.a.s.c.Overseer Overseer (id=98535344283975685-127.0.0.1:40523_u_v-n_0000000000) starting
[junit4] 2> 795324 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:40523_u_v ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40523_u_v
[junit4] 2> 795441 INFO (zkCallback-86-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 796229 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:40523_u_v ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/control-001/cores
[junit4] 2> 796230 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:40523_u_v ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 796351 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 796518 INFO (OverseerStateUpdate-98535344283975685-127.0.0.1:40523_u_v-n_0000000000) [n:127.0.0.1:40523_u_v ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 797708 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 797872 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 798588 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 798694 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection control_collection, trusted=true
[junit4] 2> 798695 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 798752 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/control-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/control-001/cores/collection1/data/]
[junit4] 2> 798752 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@424a7000
[junit4] 2> 798770 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=99.19921875, floorSegmentMB=1.33203125, forceMergeDeletesPctAllowed=22.370001199875293, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7620034299524526
[junit4] 2> 798851 WARN (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 799028 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 799028 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 799029 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 799029 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 799130 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.47872191317199597]
[junit4] 2> 799130 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3e29465e[collection1] main]
[junit4] 2> 799131 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 799132 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 799132 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 799170 INFO (searcherExecutor-290-thread-1-processing-n:127.0.0.1:40523_u_v x:collection1 c:control_collection) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3e29465e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 799174 INFO (coreLoadExecutor-289-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v c:control_collection x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576565516262178816
[junit4] 2> 799390 INFO (coreZkRegister-282-thread-1-processing-n:127.0.0.1:40523_u_v x:collection1 c:control_collection) [n:127.0.0.1:40523_u_v c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 799390 INFO (coreZkRegister-282-thread-1-processing-n:127.0.0.1:40523_u_v x:collection1 c:control_collection) [n:127.0.0.1:40523_u_v c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 799391 INFO (coreZkRegister-282-thread-1-processing-n:127.0.0.1:40523_u_v x:collection1 c:control_collection) [n:127.0.0.1:40523_u_v c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:40523/u_v/collection1/
[junit4] 2> 799391 INFO (coreZkRegister-282-thread-1-processing-n:127.0.0.1:40523_u_v x:collection1 c:control_collection) [n:127.0.0.1:40523_u_v c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 799391 INFO (coreZkRegister-282-thread-1-processing-n:127.0.0.1:40523_u_v x:collection1 c:control_collection) [n:127.0.0.1:40523_u_v c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:40523/u_v/collection1/ has no replicas
[junit4] 2> 799391 INFO (coreZkRegister-282-thread-1-processing-n:127.0.0.1:40523_u_v x:collection1 c:control_collection) [n:127.0.0.1:40523_u_v c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 799409 INFO (coreZkRegister-282-thread-1-processing-n:127.0.0.1:40523_u_v x:collection1 c:control_collection) [n:127.0.0.1:40523_u_v c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:40523/u_v/collection1/ shard1
[junit4] 2> 799551 INFO (coreZkRegister-282-thread-1-processing-n:127.0.0.1:40523_u_v x:collection1 c:control_collection) [n:127.0.0.1:40523_u_v c:control_collection s:shard1 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 799715 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 799716 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46684/solr ready
[junit4] 2> 799716 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 799716 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Creating collection1 with stateFormat=2
[junit4] 2> 800337 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-1-001/cores/collection1
[junit4] 2> 800351 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-1-001
[junit4] 2> 800368 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 800485 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4cd74a7f{/u_v,null,AVAILABLE}
[junit4] 2> 800488 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1e1158a1{SSL,[ssl, http/1.1]}{127.0.0.1:34151}
[junit4] 2> 800488 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.Server Started @812874ms
[junit4] 2> 800488 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/u_v, hostPort=34151, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-1-001/cores}
[junit4] 2> 800488 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 800497 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 800498 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 800498 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 800498 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-23T23:14:02.315Z
[junit4] 2> 800583 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e115fa1de0008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 800599 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 800599 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-1-001/solr.xml
[junit4] 2> 800666 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 800741 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46684/solr
[junit4] 2> 801167 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 801185 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:34151_u_v ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 801187 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:34151_u_v ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34151_u_v
[junit4] 2> 801200 INFO (zkCallback-86-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 801285 INFO (zkCallback-96-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 801366 INFO (zkCallback-90-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 801631 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:34151_u_v ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-1-001/cores
[junit4] 2> 801631 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:34151_u_v ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 801752 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 801753 INFO (OverseerStateUpdate-98535344283975685-127.0.0.1:40523_u_v-n_0000000000) [n:127.0.0.1:40523_u_v ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 801866 INFO (zkCallback-96-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 802861 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 802971 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 803878 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 804024 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 804024 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 804024 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-1-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-1-001/cores/collection1/data/]
[junit4] 2> 804025 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@424a7000
[junit4] 2> 804038 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=99.19921875, floorSegmentMB=1.33203125, forceMergeDeletesPctAllowed=22.370001199875293, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7620034299524526
[junit4] 2> 804087 WARN (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 804416 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 804416 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 804417 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 804417 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 804520 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.47872191317199597]
[junit4] 2> 804521 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5325838b[collection1] main]
[junit4] 2> 804550 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 804550 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 804550 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 804552 INFO (searcherExecutor-301-thread-1-processing-n:127.0.0.1:34151_u_v x:collection1 c:collection1) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@5325838b[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 804552 INFO (coreLoadExecutor-300-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576565521901420544
[junit4] 2> 804746 INFO (coreZkRegister-295-thread-1-processing-n:127.0.0.1:34151_u_v x:collection1 c:collection1) [n:127.0.0.1:34151_u_v c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 804746 INFO (coreZkRegister-295-thread-1-processing-n:127.0.0.1:34151_u_v x:collection1 c:collection1) [n:127.0.0.1:34151_u_v c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 804746 INFO (coreZkRegister-295-thread-1-processing-n:127.0.0.1:34151_u_v x:collection1 c:collection1) [n:127.0.0.1:34151_u_v c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:34151/u_v/collection1/
[junit4] 2> 804746 INFO (coreZkRegister-295-thread-1-processing-n:127.0.0.1:34151_u_v x:collection1 c:collection1) [n:127.0.0.1:34151_u_v c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 804746 INFO (coreZkRegister-295-thread-1-processing-n:127.0.0.1:34151_u_v x:collection1 c:collection1) [n:127.0.0.1:34151_u_v c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:34151/u_v/collection1/ has no replicas
[junit4] 2> 804746 INFO (coreZkRegister-295-thread-1-processing-n:127.0.0.1:34151_u_v x:collection1 c:collection1) [n:127.0.0.1:34151_u_v c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 804784 INFO (coreZkRegister-295-thread-1-processing-n:127.0.0.1:34151_u_v x:collection1 c:collection1) [n:127.0.0.1:34151_u_v c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:34151/u_v/collection1/ shard2
[junit4] 2> 804814 INFO (zkCallback-96-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 805107 INFO (zkCallback-96-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 805109 INFO (coreZkRegister-295-thread-1-processing-n:127.0.0.1:34151_u_v x:collection1 c:collection1) [n:127.0.0.1:34151_u_v c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 805257 INFO (zkCallback-96-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 806136 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-2-001/cores/collection1
[junit4] 2> 806137 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-2-001
[junit4] 2> 806172 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 806269 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@26e3b262{/u_v,null,AVAILABLE}
[junit4] 2> 806270 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@18088fcd{SSL,[ssl, http/1.1]}{127.0.0.1:41648}
[junit4] 2> 806270 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.Server Started @818656ms
[junit4] 2> 806270 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/u_v, hostPort=41648, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-2-001/cores}
[junit4] 2> 806270 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 806270 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 806271 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 806271 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 806271 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-23T23:14:08.088Z
[junit4] 2> 806334 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 806334 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-2-001/solr.xml
[junit4] 2> 806412 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 806413 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46684/solr
[junit4] 2> 806428 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e115fa1de000c, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 806479 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:41648_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 806493 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:41648_u_v ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 806494 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:41648_u_v ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41648_u_v
[junit4] 2> 806495 INFO (zkCallback-90-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 806495 INFO (zkCallback-86-thread-1-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 806505 INFO (zkCallback-96-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 806558 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 806863 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:41648_u_v ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-2-001/cores
[junit4] 2> 806863 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:41648_u_v ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 806912 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 806968 INFO (OverseerStateUpdate-98535344283975685-127.0.0.1:40523_u_v-n_0000000000) [n:127.0.0.1:40523_u_v ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard1
[junit4] 2> 807168 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 807172 INFO (zkCallback-96-thread-1-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 808169 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 808457 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 810552 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 810922 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 810922 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 810922 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-2-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-2-001/cores/collection1/data/]
[junit4] 2> 810922 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@424a7000
[junit4] 2> 811034 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=99.19921875, floorSegmentMB=1.33203125, forceMergeDeletesPctAllowed=22.370001199875293, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7620034299524526
[junit4] 2> 811067 WARN (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 822258 WARN (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]-SendThread(127.0.0.1:46684)) [n:127.0.0.1:34151_u_v ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 15078ms for sessionid 0x15e115fa1de000a
[junit4] 2> 822259 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e115fa1de000a, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 822259 WARN (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]-SendThread(127.0.0.1:46684)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 12420ms for sessionid 0x15e115fa1de0006
[junit4] 2> 822429 WARN (zkCallback-96-thread-4-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@64a62735 name: ZooKeeperConnection Watcher:127.0.0.1:46684/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 822429 WARN (zkCallback-96-thread-4-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 822474 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 822474 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 822475 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 822475 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 822490 WARN (zkCallback-90-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@e0ffb6b name: ZooKeeperConnection Watcher:127.0.0.1:46684/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 822526 WARN (zkCallback-90-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 822526 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.47872191317199597]
[junit4] 2> 822526 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@19e71e39[collection1] main]
[junit4] 2> 822608 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 822625 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 822625 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 822627 INFO (searcherExecutor-312-thread-1-processing-n:127.0.0.1:41648_u_v x:collection1 c:collection1) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@19e71e39[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 822627 INFO (coreLoadExecutor-311-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576565540854431744
[junit4] 2> 822686 INFO (coreZkRegister-306-thread-1-processing-n:127.0.0.1:41648_u_v x:collection1 c:collection1) [n:127.0.0.1:41648_u_v c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 822686 INFO (coreZkRegister-306-thread-1-processing-n:127.0.0.1:41648_u_v x:collection1 c:collection1) [n:127.0.0.1:41648_u_v c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 822686 INFO (coreZkRegister-306-thread-1-processing-n:127.0.0.1:41648_u_v x:collection1 c:collection1) [n:127.0.0.1:41648_u_v c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:41648/u_v/collection1/
[junit4] 2> 822686 INFO (coreZkRegister-306-thread-1-processing-n:127.0.0.1:41648_u_v x:collection1 c:collection1) [n:127.0.0.1:41648_u_v c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 822686 INFO (coreZkRegister-306-thread-1-processing-n:127.0.0.1:41648_u_v x:collection1 c:collection1) [n:127.0.0.1:41648_u_v c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.SyncStrategy https://127.0.0.1:41648/u_v/collection1/ has no replicas
[junit4] 2> 822686 INFO (coreZkRegister-306-thread-1-processing-n:127.0.0.1:41648_u_v x:collection1 c:collection1) [n:127.0.0.1:41648_u_v c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 822688 INFO (coreZkRegister-306-thread-1-processing-n:127.0.0.1:41648_u_v x:collection1 c:collection1) [n:127.0.0.1:41648_u_v c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:41648/u_v/collection1/ shard1
[junit4] 2> 822739 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 822887 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 822917 INFO (coreZkRegister-306-thread-1-processing-n:127.0.0.1:41648_u_v x:collection1 c:collection1) [n:127.0.0.1:41648_u_v c:collection1 s:shard1 r:core_node2 x:collection1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 823080 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 823392 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-3-001/cores/collection1
[junit4] 2> 823486 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-3-001
[junit4] 2> 823488 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 823514 INFO (zkCallback-96-thread-4-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 823624 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@395e6547{/u_v,null,AVAILABLE}
[junit4] 2> 823625 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@31bcbc5f{SSL,[ssl, http/1.1]}{127.0.0.1:42485}
[junit4] 2> 823625 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.Server Started @836011ms
[junit4] 2> 823625 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/u_v, hostPort=42485, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-3-001/cores}
[junit4] 2> 823625 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 823625 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 823625 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 823625 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 823625 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-23T23:14:25.442Z
[junit4] 2> 823758 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 823758 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-3-001/solr.xml
[junit4] 2> 823807 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 823808 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46684/solr
[junit4] 2> 824026 WARN (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]-SendThread(127.0.0.1:46684)) [ ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x15e115fa1de0006 has expired
[junit4] 2> 824047 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:42485_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 824034 WARN (zkCallback-90-thread-2) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@e0ffb6b name: ZooKeeperConnection Watcher:127.0.0.1:46684/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 824051 WARN (zkCallback-90-thread-2) [ ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 824052 WARN (zkCallback-90-thread-2) [ ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 824053 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:42485_u_v ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 824124 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:42485_u_v ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42485_u_v
[junit4] 2> 824132 INFO (zkCallback-90-thread-2) [ ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 824238 INFO (zkCallback-86-thread-2-processing-n:127.0.0.1:40523_u_v) [n:127.0.0.1:40523_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 824238 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 824250 INFO (zkCallback-90-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 824311 INFO (zkCallback-108-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 824315 INFO (zkCallback-96-thread-4-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 824379 INFO (zkCallback-90-thread-2) [ ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
[junit4] 2> 824380 INFO (zkCallback-90-thread-2) [ ] o.a.s.c.c.ConnectionManager Connected:true
[junit4] 2> 824934 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:42485_u_v ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-3-001/cores
[junit4] 2> 824934 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [n:127.0.0.1:42485_u_v ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 825021 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 825023 INFO (OverseerStateUpdate-98535344283975685-127.0.0.1:40523_u_v-n_0000000000) [n:127.0.0.1:40523_u_v ] o.a.s.c.o.ReplicaMutator Assigning new node to shard shard=shard2
[junit4] 2> 825125 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 825125 INFO (zkCallback-108-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 825131 INFO (zkCallback-96-thread-4-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 826135 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 826171 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 826637 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 826705 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from collection collection1, trusted=true
[junit4] 2> 826706 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 826706 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-3-001/cores/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-3-001/cores/collection1/data/]
[junit4] 2> 826706 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.c.JmxMonitoredMap JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@424a7000
[junit4] 2> 826707 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=36, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=99.19921875, floorSegmentMB=1.33203125, forceMergeDeletesPctAllowed=22.370001199875293, segmentsPerTier=23.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7620034299524526
[junit4] 2> 826789 WARN (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 827049 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 827049 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 827050 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 827050 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 827115 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=20, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.47872191317199597]
[junit4] 2> 827115 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3c42453d[collection1] main]
[junit4] 2> 827116 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 827116 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 827116 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 827118 INFO (searcherExecutor-323-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 c:collection1) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@3c42453d[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 827130 INFO (coreLoadExecutor-322-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v c:collection1 x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576565545576169472
[junit4] 2> 827164 INFO (coreZkRegister-317-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 c:collection1) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.ZkController Core needs to recover:collection1
[junit4] 2> 827180 INFO (updateExecutor-105-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 827236 INFO (recoveryExecutor-106-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 827245 INFO (recoveryExecutor-106-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 827245 INFO (recoveryExecutor-106-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1]
[junit4] 2> 827245 INFO (recoveryExecutor-106-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 827245 INFO (recoveryExecutor-106-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Publishing state of core [collection1] as recovering, leader is [https://127.0.0.1:34151/u_v/collection1/] and I am [https://127.0.0.1:42485/u_v/collection1/]
[junit4] 2> 827248 INFO (zkCallback-102-thread-1-processing-n:127.0.0.1:41648_u_v) [n:127.0.0.1:41648_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 827248 INFO (zkCallback-108-thread-1-processing-n:127.0.0.1:42485_u_v) [n:127.0.0.1:42485_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 827252 INFO (zkCallback-96-thread-4-processing-n:127.0.0.1:34151_u_v) [n:127.0.0.1:34151_u_v ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 827277 INFO (recoveryExecutor-106-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:34151/u_v]; [WaitForState: action=PREPRECOVERY&core=collection1&nodeName=127.0.0.1:42485_u_v&coreNodeName=core_node3&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 827966 INFO (qtp1036511527-632) [n:127.0.0.1:34151_u_v ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 827967 INFO (qtp1036511527-632) [n:127.0.0.1:34151_u_v ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see collection1 (shard2 of collection1) have state: recovering
[junit4] 2> 827967 INFO (qtp1036511527-632) [n:127.0.0.1:34151_u_v ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:42485_u_v, coreNodeName=core_node3, onlyIfActiveCheckResult=false, nodeProps: core_node3:{"core":"collection1","base_url":"https://127.0.0.1:42485/u_v","node_name":"127.0.0.1:42485_u_v","state":"recovering"}
[junit4] 2> 827967 INFO (qtp1036511527-632) [n:127.0.0.1:34151_u_v ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node3, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
[junit4] 2> 827967 INFO (qtp1036511527-632) [n:127.0.0.1:34151_u_v ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:42485_u_v&onlyIfLeaderActive=true&core=collection1&coreNodeName=core_node3&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 828556 INFO (recoveryExecutor-106-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [https://127.0.0.1:34151/u_v/collection1/] - recoveringAfterStartup=[true]
[junit4] 2> 828621 INFO (recoveryExecutor-106-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.PeerSync PeerSync: core=collection1 url=https://127.0.0.1:42485/u_v START replicas=[https://127.0.0.1:34151/u_v/collection1/] nUpdates=100
[junit4] 2> 828810 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-4-001/cores/collection1
[junit4] 2> 828811 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-4-001
[junit4] 2> 828859 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 828990 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6ebb0ae5{/u_v,null,AVAILABLE}
[junit4] 2> 828991 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@dcf128d{SSL,[ssl, http/1.1]}{127.0.0.1:43261}
[junit4] 2> 828991 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.e.j.s.Server Started @841377ms
[junit4] 2> 828991 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/tempDir-001/jetty4, solrconfig=solrconfig.xml, hostContext=/u_v, hostPort=43261, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerConcurrent_392477BBD94A556C-001/shard-4-001/cores}
[junit4] 2> 828991 ERROR (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 828991 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 828991 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 828991 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 828991 INFO (TEST-TestSolrConfigHandlerConcurrent.test-seed#[392477BBD94A556C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-23T23:14:30.808Z
[junit4] 2> 829338 INFO (qtp1036511527-634) [n:127.0.0.1:34151_u_v c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:183.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 829339 INFO (qtp1036511527-634) [n:127.0.0.1:34151_u_v c:collection1 s:shard2 r:core_node1 x:collection1] o.a.s.c.S.Request [collection1] webapp=/u_v path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=185
[junit4] 2> 829497 INFO (recoveryExecutor-106-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.IndexFingerprint IndexFingerprint millis:156.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 829518 INFO (recoveryExecutor-106-thread-1-processing-n:127.0.0.1:42485_u_v x:collection1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:42485_u_v c:collection1 s:shard2 r:core_node3 x:collection1] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 829518 INFO (recoveryExecutor-106-thread-1-pro
[...truncated too long message...]
Consume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1872485 INFO (qtp100522364-3984) [n:127.0.0.1:42453_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=badactioncollection&action=BADACTION&numShards=2&wt=javabin&version=2} status=400 QTime=12
[junit4] 2> 1872485 INFO (TEST-CollectionsAPIDistributedZkTest.testBadActionNames-seed#[392477BBD94A556C]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBadActionNames
[junit4] 2> 1872649 INFO (jetty-closer-616-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@235eea65{SSL,[ssl, http/1.1]}{127.0.0.1:40194}
[junit4] 2> 1872666 INFO (jetty-closer-616-thread-3) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@79bdabf0{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1872667 INFO (jetty-closer-616-thread-4) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@339a2991{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1872667 INFO (jetty-closer-616-thread-4) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1790950748
[junit4] 2> 1872667 INFO (jetty-closer-616-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 1872667 INFO (jetty-closer-616-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 1872696 INFO (jetty-closer-616-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1606842283
[junit4] 2> 1872696 INFO (jetty-closer-616-thread-3) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=519126008
[junit4] 2> 1872696 INFO (jetty-closer-616-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 1872696 INFO (jetty-closer-616-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 1872696 INFO (jetty-closer-616-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 1872696 INFO (jetty-closer-616-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 1872696 INFO (jetty-closer-616-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 1872696 INFO (jetty-closer-616-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 1872702 INFO (jetty-closer-616-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 1872712 INFO (jetty-closer-616-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@48cb9c01{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1872712 INFO (jetty-closer-616-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=236338165
[junit4] 2> 1872712 INFO (jetty-closer-616-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 1872712 INFO (jetty-closer-616-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 1872712 INFO (jetty-closer-616-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 1872715 INFO (zkCallback-637-thread-4-processing-n:127.0.0.1:42453_solr) [n:127.0.0.1:42453_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1872728 INFO (jetty-closer-616-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4e012c91{/solr,null,UNAVAILABLE}
[junit4] 2> 1872734 INFO (zkCallback-636-thread-7-processing-n:127.0.0.1:38426_solr) [n:127.0.0.1:38426_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1872734 INFO (zkCallback-634-thread-2-processing-n:127.0.0.1:42289_solr) [n:127.0.0.1:42289_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1872749 WARN (zkCallback-637-thread-7-processing-n:127.0.0.1:42453_solr) [n:127.0.0.1:42453_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1872761 INFO (jetty-closer-616-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4d402a14{/solr,null,UNAVAILABLE}
[junit4] 2> 1872765 INFO (jetty-closer-616-thread-3) [ ] o.a.s.c.Overseer Overseer (id=98535406681718794-127.0.0.1:42289_solr-n_0000000000) closing
[junit4] 2> 1872765 INFO (zkCallback-634-thread-3-processing-n:127.0.0.1:42289_solr) [n:127.0.0.1:42289_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1)
[junit4] 2> 1872765 INFO (OverseerStateUpdate-98535406681718794-127.0.0.1:42289_solr-n_0000000000) [n:127.0.0.1:42289_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42289_solr
[junit4] 2> 1874260 WARN (zkCallback-636-thread-7-processing-n:127.0.0.1:38426_solr) [n:127.0.0.1:38426_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1874260 INFO (jetty-closer-616-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@942c6d5{/solr,null,UNAVAILABLE}
[junit4] 2> 1874321 WARN (zkCallback-634-thread-3-processing-n:127.0.0.1:42289_solr) [n:127.0.0.1:42289_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1874325 INFO (jetty-closer-616-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@192550e1{/solr,null,UNAVAILABLE}
[junit4] 2> 1874326 ERROR (SUITE-CollectionsAPIDistributedZkTest-seed#[392477BBD94A556C]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1874326 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[392477BBD94A556C]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44640 44640
[junit4] 2> 1874328 INFO (Thread-1154) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:44640 44640
[junit4] 2> 1874329 WARN (Thread-1154) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 139 /solr/clusterstate.json
[junit4] 2> 4 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 99 /solr/collections
[junit4] 2> 4 /solr/live_nodes
[junit4] 2>
[junit4] 2> 1874390 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[392477BBD94A556C]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_392477BBD94A556C-001
[junit4] 2> Aug 23, 2017 11:31:56 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=30710, maxDocsPerChunk=856, blockSize=1), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=30710, blockSize=1)), sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=sl, timezone=Europe/Bucharest
[junit4] 2> NOTE: Linux 4.4.0-83-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=251825504,total=489160704
[junit4] 2> NOTE: All tests run in this JVM: [BasicFunctionalityTest, TestRawResponseWriter, CollectionStateFormat2Test, TestManagedStopFilterFactory, TestPayloadScoreQParserPlugin, TestLMDirichletSimilarityFactory, SegmentsInfoRequestHandlerTest, RankQueryTest, TestCSVResponseWriter, TestQueryUtils, TestSolrCoreSnapshots, TestInPlaceUpdatesStandalone, TestMaxScoreQueryParser, TestElisionMultitermQuery, SolrJmxReporterCloudTest, DistributedDebugComponentTest, DistributedQueryComponentOptimizationTest, TestQueryTypes, ZkStateReaderTest, TestSweetSpotSimilarityFactory, SaslZkACLProviderTest, CdcrBootstrapTest, HdfsWriteToMultipleCollectionsTest, TestUpdate, TestShardHandlerFactory, DirectSolrSpellCheckerTest, OverseerModifyCollectionTest, IndexSchemaTest, TestLRUCache, TestRestManager, SystemInfoHandlerTest, DocValuesTest, MultiTermTest, ForceLeaderTest, TestStressCloudBlindAtomicUpdates, TestRuleBasedAuthorizationPlugin, TestTrackingShardHandlerFactory, TestConfigSets, TestCloudDeleteByQuery, TestCollectionAPIs, CdcrRequestHandlerTest, SuggesterFSTTest, MoveReplicaTest, SimplePostToolTest, TestImpersonationWithHadoopAuth, TestDistributedStatsComponentCardinality, DistributedMLTComponentTest, TestHashPartitioner, AnalysisErrorHandlingTest, TestRTGBase, TestTrieFacet, SparseHLLTest, BlockJoinFacetDistribTest, TestCoreContainer, TestChildDocTransformer, CollectionsAPIDistributedZkTest]
[junit4] Completed [199/713 (2!)] on J1 in 131.64s, 20 tests, 1 failure <<< FAILURES!
[...truncated 45198 lines...]
[JENKINS] Lucene-Solr-Tests-6.6 - Build # 35 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.6/35/
3 tests failed.
FAILED: org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
Stack Trace:
java.lang.AssertionError: Something is broken in the assert for no shards using the same indexDir - probably something was changed in the attributes published in the MBean of SolrCore : {}
at __randomizedtesting.SeedInfo.seed([91A83F8C007FE8E3:D9DD4B38064CC776]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.checkNoTwoShardsUseTheSameIndexDir(CollectionsAPIDistributedZkTest.java:646)
at org.apache.solr.cloud.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:524)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.core.TestJmxIntegration
Error Message:
No JMX server found by monitor map
Stack Trace:
java.lang.AssertionError: No JMX server found by monitor map
at __randomizedtesting.SeedInfo.seed([91A83F8C007FE8E3]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.core.TestJmxIntegration.beforeClass(TestJmxIntegration.java:75)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:847)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.MetricsHandlerTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([91A83F8C007FE8E3:19FC0056AE83851B]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.junit.Assert.assertNotNull(Assert.java:537)
at org.apache.solr.handler.admin.MetricsHandlerTest.test(MetricsHandlerTest.java:54)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11426 lines...]
[junit4] Suite: org.apache.solr.cloud.CollectionsAPIDistributedZkTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/init-core-data-001
[junit4] 2> 627626 WARN (SUITE-CollectionsAPIDistributedZkTest-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 627627 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 627628 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 627629 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001
[junit4] 2> 627629 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 627629 INFO (Thread-905) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 627629 INFO (Thread-905) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 627635 ERROR (Thread-905) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 627729 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:41313
[junit4] 2> 627742 INFO (jetty-launcher-407-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 627743 INFO (jetty-launcher-407-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 627747 INFO (jetty-launcher-407-thread-3) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 627749 INFO (jetty-launcher-407-thread-4) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 627750 INFO (jetty-launcher-407-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7cadf2ef{/solr,null,AVAILABLE}
[junit4] 2> 627752 INFO (jetty-launcher-407-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5d7742c4{/solr,null,AVAILABLE}
[junit4] 2> 627754 INFO (jetty-launcher-407-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7c35c048{SSL,[ssl, http/1.1]}{127.0.0.1:55021}
[junit4] 2> 627754 INFO (jetty-launcher-407-thread-4) [ ] o.e.j.s.Server Started @631732ms
[junit4] 2> 627754 INFO (jetty-launcher-407-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55021}
[junit4] 2> 627754 ERROR (jetty-launcher-407-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 627754 INFO (jetty-launcher-407-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 627754 INFO (jetty-launcher-407-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 627754 INFO (jetty-launcher-407-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 627754 INFO (jetty-launcher-407-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-22T05:04:27.423Z
[junit4] 2> 627756 INFO (jetty-launcher-407-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@64ece2bc{/solr,null,AVAILABLE}
[junit4] 2> 627757 INFO (jetty-launcher-407-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@24f05543{SSL,[ssl, http/1.1]}{127.0.0.1:40278}
[junit4] 2> 627757 INFO (jetty-launcher-407-thread-2) [ ] o.e.j.s.Server Started @631735ms
[junit4] 2> 627757 INFO (jetty-launcher-407-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40278}
[junit4] 2> 627757 ERROR (jetty-launcher-407-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 627757 INFO (jetty-launcher-407-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 627757 INFO (jetty-launcher-407-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 627757 INFO (jetty-launcher-407-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 627757 INFO (jetty-launcher-407-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-22T05:04:27.426Z
[junit4] 2> 627759 INFO (jetty-launcher-407-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3baf061{SSL,[ssl, http/1.1]}{127.0.0.1:34789}
[junit4] 2> 627759 INFO (jetty-launcher-407-thread-3) [ ] o.e.j.s.Server Started @631737ms
[junit4] 2> 627759 INFO (jetty-launcher-407-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34789}
[junit4] 2> 627759 ERROR (jetty-launcher-407-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 627759 INFO (jetty-launcher-407-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 627759 INFO (jetty-launcher-407-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 627759 INFO (jetty-launcher-407-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 627759 INFO (jetty-launcher-407-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-22T05:04:27.428Z
[junit4] 2> 627766 INFO (jetty-launcher-407-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@356de790{/solr,null,AVAILABLE}
[junit4] 2> 627774 INFO (jetty-launcher-407-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@698f1164{SSL,[ssl, http/1.1]}{127.0.0.1:35418}
[junit4] 2> 627774 INFO (jetty-launcher-407-thread-1) [ ] o.e.j.s.Server Started @631752ms
[junit4] 2> 627774 INFO (jetty-launcher-407-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35418}
[junit4] 2> 627774 ERROR (jetty-launcher-407-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 627774 INFO (jetty-launcher-407-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 6.6.1
[junit4] 2> 627774 INFO (jetty-launcher-407-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 627774 INFO (jetty-launcher-407-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 627774 INFO (jetty-launcher-407-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-22T05:04:27.443Z
[junit4] 2> 627776 INFO (jetty-launcher-407-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 627779 INFO (jetty-launcher-407-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 627803 INFO (jetty-launcher-407-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 627804 INFO (jetty-launcher-407-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41313/solr
[junit4] 2> 627806 INFO (jetty-launcher-407-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 627818 INFO (jetty-launcher-407-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 627835 INFO (jetty-launcher-407-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41313/solr
[junit4] 2> 627847 INFO (jetty-launcher-407-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 627854 INFO (jetty-launcher-407-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 627855 INFO (jetty-launcher-407-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41313/solr
[junit4] 2> 627890 INFO (jetty-launcher-407-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 627902 INFO (jetty-launcher-407-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41313/solr
[junit4] 2> 627929 INFO (jetty-launcher-407-thread-2) [n:127.0.0.1:40278_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 627937 INFO (jetty-launcher-407-thread-2) [n:127.0.0.1:40278_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40278_solr
[junit4] 2> 627938 INFO (jetty-launcher-407-thread-4) [n:127.0.0.1:55021_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 627939 INFO (jetty-launcher-407-thread-4) [n:127.0.0.1:55021_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55021_solr
[junit4] 2> 627939 INFO (jetty-launcher-407-thread-2) [n:127.0.0.1:40278_solr ] o.a.s.c.Overseer Overseer (id=98525398125903879-127.0.0.1:40278_solr-n_0000000000) starting
[junit4] 2> 627946 INFO (zkCallback-421-thread-1-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 627950 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 627957 INFO (jetty-launcher-407-thread-3) [n:127.0.0.1:34789_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 627961 INFO (jetty-launcher-407-thread-3) [n:127.0.0.1:34789_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 627962 INFO (jetty-launcher-407-thread-3) [n:127.0.0.1:34789_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34789_solr
[junit4] 2> 627963 INFO (zkCallback-421-thread-1-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 627964 INFO (jetty-launcher-407-thread-1) [n:127.0.0.1:35418_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 627967 INFO (jetty-launcher-407-thread-1) [n:127.0.0.1:35418_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 627968 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 627969 INFO (jetty-launcher-407-thread-1) [n:127.0.0.1:35418_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35418_solr
[junit4] 2> 627970 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 627970 INFO (zkCallback-421-thread-1-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 627975 INFO (zkCallback-429-thread-1-processing-n:127.0.0.1:34789_solr) [n:127.0.0.1:34789_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 627994 INFO (jetty-launcher-407-thread-2) [n:127.0.0.1:40278_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40278_solr
[junit4] 2> 627996 INFO (zkCallback-429-thread-1-processing-n:127.0.0.1:34789_solr) [n:127.0.0.1:34789_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 627996 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 627997 INFO (zkCallback-421-thread-1-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 628010 INFO (zkCallback-431-thread-2-processing-n:127.0.0.1:35418_solr) [n:127.0.0.1:35418_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 628135 INFO (jetty-launcher-407-thread-3) [n:127.0.0.1:34789_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node3/.
[junit4] 2> 628141 INFO (jetty-launcher-407-thread-2) [n:127.0.0.1:40278_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node2/.
[junit4] 2> 628142 INFO (jetty-launcher-407-thread-4) [n:127.0.0.1:55021_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node4/.
[junit4] 2> 628283 INFO (jetty-launcher-407-thread-1) [n:127.0.0.1:35418_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node1/.
[junit4] 2> 628560 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 628561 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41313/solr ready
[junit4] 2> 628623 INFO (TEST-CollectionsAPIDistributedZkTest.testMissingRequiredParameters-seed#[91A83F8C007FE8E3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMissingRequiredParameters
[junit4] 2> 628624 INFO (TEST-CollectionsAPIDistributedZkTest.testMissingRequiredParameters-seed#[91A83F8C007FE8E3]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 628704 INFO (qtp1377695483-3017) [n:127.0.0.1:55021_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 628704 ERROR (qtp1377695483-3017) [n:127.0.0.1:55021_solr ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: Missing required parameter: name
[junit4] 2> at org.apache.solr.common.params.RequiredSolrParams.getParams(RequiredSolrParams.java:90)
[junit4] 2> at org.apache.solr.common.params.SolrParams.getAll(SolrParams.java:462)
[junit4] 2> at org.apache.solr.common.params.SolrParams.getAll(SolrParams.java:479)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.lambda$static$0(CollectionsHandler.java:396)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation.execute(CollectionsHandler.java:922)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:228)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:215)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:173)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:748)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:729)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:510)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:136)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1699)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:462)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:202)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 628704 INFO (qtp1377695483-3017) [n:127.0.0.1:55021_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=CREATE&numShards=2&wt=javabin&version=2} status=400 QTime=0
[junit4] 2> 628705 INFO (TEST-CollectionsAPIDistributedZkTest.testMissingRequiredParameters-seed#[91A83F8C007FE8E3]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testMissingRequiredParameters
[junit4] 2> 628729 INFO (TEST-CollectionsAPIDistributedZkTest.addReplicaTest-seed#[91A83F8C007FE8E3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting addReplicaTest
[junit4] 2> 628730 INFO (TEST-CollectionsAPIDistributedZkTest.addReplicaTest-seed#[91A83F8C007FE8E3]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 628735 INFO (qtp1377695483-3019) [n:127.0.0.1:55021_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=4&name=addReplicaColl&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 628737 INFO (OverseerThreadFactory-1438-thread-1-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.CreateCollectionCmd Create collection addReplicaColl
[junit4] 2> 628968 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica2&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2
[junit4] 2> 628968 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 628974 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica1&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2
[junit4] 2> 628975 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 628977 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica2&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2
[junit4] 2> 628978 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 628979 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica1&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2
[junit4] 2> 628979 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 629086 INFO (zkCallback-431-thread-2-processing-n:127.0.0.1:35418_solr) [n:127.0.0.1:35418_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 629086 INFO (zkCallback-421-thread-2-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 629086 INFO (zkCallback-429-thread-1-processing-n:127.0.0.1:34789_solr) [n:127.0.0.1:34789_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 629086 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630005 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 630010 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 630020 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 630028 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 630035 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.s.IndexSchema [addReplicaColl_shard1_replica2] Schema name=minimal
[junit4] 2> 630078 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 630078 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard1_replica2' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 630078 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 630078 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.SolrCore [[addReplicaColl_shard1_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node4/addReplicaColl_shard1_replica2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node4/./addReplicaColl_shard1_replica2/data/]
[junit4] 2> 630081 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.s.IndexSchema [addReplicaColl_shard1_replica1] Schema name=minimal
[junit4] 2> 630100 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 630100 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard1_replica1' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 630100 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 630100 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.SolrCore [[addReplicaColl_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node3/addReplicaColl_shard1_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node3/./addReplicaColl_shard1_replica1/data/]
[junit4] 2> 630101 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.s.IndexSchema [addReplicaColl_shard2_replica2] Schema name=minimal
[junit4] 2> 630103 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 630103 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard2_replica2' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 630104 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 630104 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.SolrCore [[addReplicaColl_shard2_replica2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node1/addReplicaColl_shard2_replica2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node1/./addReplicaColl_shard2_replica2/data/]
[junit4] 2> 630106 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.s.IndexSchema [addReplicaColl_shard2_replica1] Schema name=minimal
[junit4] 2> 630108 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 630108 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard2_replica1' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 630108 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 630108 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.SolrCore [[addReplicaColl_shard2_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node2/addReplicaColl_shard2_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node2/./addReplicaColl_shard2_replica1/data/]
[junit4] 2> 630273 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 630273 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 630278 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 630278 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 630281 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@763b33b1[addReplicaColl_shard1_replica2] main]
[junit4] 2> 630285 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 630286 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 630286 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 630287 INFO (searcherExecutor-1447-thread-1-processing-n:127.0.0.1:55021_solr x:addReplicaColl_shard1_replica2 s:shard1 c:addReplicaColl) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.SolrCore [addReplicaColl_shard1_replica2] Registered new searcher Searcher@763b33b1[addReplicaColl_shard1_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 630287 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576406372797382656
[junit4] 2> 630291 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 630291 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 630292 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 630292 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 630296 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.s.SolrIndexSearcher Opening [Searcher@14c59fce[addReplicaColl_shard2_replica2] main]
[junit4] 2> 630296 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 630296 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 630297 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 630297 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 630298 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5e2177e5[addReplicaColl_shard2_replica1] main]
[junit4] 2> 630298 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 630298 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 630299 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 630299 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 630300 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6a46d6ef[addReplicaColl_shard1_replica1] main]
[junit4] 2> 630300 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 630300 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 630301 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 630301 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 630302 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576406372813111296
[junit4] 2> 630302 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 630302 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 630306 INFO (searcherExecutor-1449-thread-1-processing-n:127.0.0.1:35418_solr x:addReplicaColl_shard2_replica2 s:shard2 c:addReplicaColl) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.SolrCore [addReplicaColl_shard2_replica2] Registered new searcher Searcher@14c59fce[addReplicaColl_shard2_replica2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 630306 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576406372817305600
[junit4] 2> 630313 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 630313 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 630313 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 630314 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 630314 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576406372825694208
[junit4] 2> 630320 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 630320 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 630320 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:35418/solr/addReplicaColl_shard2_replica2/
[junit4] 2> 630320 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.PeerSync PeerSync: core=addReplicaColl_shard2_replica2 url=https://127.0.0.1:35418/solr START replicas=[https://127.0.0.1:40278/solr/addReplicaColl_shard2_replica1/] nUpdates=100
[junit4] 2> 630326 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.u.PeerSync PeerSync: core=addReplicaColl_shard2_replica2 url=https://127.0.0.1:35418/solr DONE. We have no versions. sync failed.
[junit4] 2> 630329 INFO (searcherExecutor-1450-thread-1-processing-n:127.0.0.1:40278_solr x:addReplicaColl_shard2_replica1 s:shard2 c:addReplicaColl) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.c.SolrCore [addReplicaColl_shard2_replica1] Registered new searcher Searcher@5e2177e5[addReplicaColl_shard2_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 630330 INFO (searcherExecutor-1448-thread-1-processing-n:127.0.0.1:34789_solr x:addReplicaColl_shard1_replica1 s:shard1 c:addReplicaColl) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.c.SolrCore [addReplicaColl_shard1_replica1] Registered new searcher Searcher@6a46d6ef[addReplicaColl_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 630344 INFO (qtp1736021994-3028) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 r:core_node2 x:addReplicaColl_shard2_replica1] o.a.s.c.S.Request [addReplicaColl_shard2_replica1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 630344 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 630344 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 630344 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 630347 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:35418/solr/addReplicaColl_shard2_replica2/ shard2
[junit4] 2> 630450 INFO (zkCallback-431-thread-2-processing-n:127.0.0.1:35418_solr) [n:127.0.0.1:35418_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630450 INFO (zkCallback-429-thread-1-processing-n:127.0.0.1:34789_solr) [n:127.0.0.1:34789_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630450 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630451 INFO (zkCallback-421-thread-2-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630498 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 630516 INFO (qtp1381576184-3042) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica2&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2} status=0 QTime=1538
[junit4] 2> 630618 INFO (zkCallback-431-thread-2-processing-n:127.0.0.1:35418_solr) [n:127.0.0.1:35418_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630618 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630618 INFO (zkCallback-429-thread-1-processing-n:127.0.0.1:34789_solr) [n:127.0.0.1:34789_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630619 INFO (zkCallback-421-thread-2-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630814 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 630814 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 630814 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:55021/solr/addReplicaColl_shard1_replica2/
[junit4] 2> 630814 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=addReplicaColl_shard1_replica2 url=https://127.0.0.1:55021/solr START replicas=[https://127.0.0.1:34789/solr/addReplicaColl_shard1_replica1/] nUpdates=100
[junit4] 2> 630814 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.u.PeerSync PeerSync: core=addReplicaColl_shard1_replica2 url=https://127.0.0.1:55021/solr DONE. We have no versions. sync failed.
[junit4] 2> 630839 INFO (qtp186477155-3010) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 r:core_node4 x:addReplicaColl_shard1_replica1] o.a.s.c.S.Request [addReplicaColl_shard1_replica1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 630839 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 630839 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 630839 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 630842 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:55021/solr/addReplicaColl_shard1_replica2/ shard1
[junit4] 2> 630944 INFO (zkCallback-431-thread-2-processing-n:127.0.0.1:35418_solr) [n:127.0.0.1:35418_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630945 INFO (zkCallback-429-thread-1-processing-n:127.0.0.1:34789_solr) [n:127.0.0.1:34789_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630945 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630945 INFO (zkCallback-421-thread-2-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 630993 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 630995 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica2&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2} status=0 QTime=2027
[junit4] 2> 631099 INFO (zkCallback-431-thread-2-processing-n:127.0.0.1:35418_solr) [n:127.0.0.1:35418_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 631099 INFO (zkCallback-429-thread-1-processing-n:127.0.0.1:34789_solr) [n:127.0.0.1:34789_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 631099 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 631099 INFO (zkCallback-421-thread-2-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 631324 INFO (qtp1736021994-3029) [n:127.0.0.1:40278_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard2_replica1&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard2&wt=javabin&version=2} status=0 QTime=2349
[junit4] 2> 631327 INFO (qtp186477155-3009) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&newCollection=true&name=addReplicaColl_shard1_replica1&action=CREATE&numShards=2&collection=addReplicaColl&shard=shard1&wt=javabin&version=2} status=0 QTime=2348
[junit4] 2> 631334 INFO (qtp1377695483-3019) [n:127.0.0.1:55021_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 631438 INFO (zkCallback-431-thread-2-processing-n:127.0.0.1:35418_solr) [n:127.0.0.1:35418_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 631438 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 631438 INFO (zkCallback-429-thread-1-processing-n:127.0.0.1:34789_solr) [n:127.0.0.1:34789_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 631439 INFO (zkCallback-421-thread-2-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 632334 INFO (qtp1377695483-3019) [n:127.0.0.1:55021_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&maxShardsPerNode=4&name=addReplicaColl&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3599
[junit4] 2> 632337 INFO (qtp1377695483-3025) [n:127.0.0.1:55021_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:34789_solr&action=ADDREPLICA&collection=addReplicaColl&shard=shard1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 632342 INFO (OverseerThreadFactory-1438-thread-2-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.AddReplicaCmd addReplica() : {
[junit4] 2> "collection":"addReplicaColl",
[junit4] 2> "node":"127.0.0.1:34789_solr",
[junit4] 2> "shard":"shard1",
[junit4] 2> "operation":"addreplica"}
[junit4] 2> 632342 INFO (OverseerThreadFactory-1438-thread-2-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:34789_solr for creating new replica
[junit4] 2> 632342 INFO (OverseerCollectionConfigSetProcessor-98525398125903879-127.0.0.1:40278_solr-n_0000000000) [n:127.0.0.1:40278_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 632347 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&name=addReplicaColl_shard1_replica3&action=CREATE&collection=addReplicaColl&shard=shard1&wt=javabin&version=2
[junit4] 2> 632450 INFO (zkCallback-431-thread-2-processing-n:127.0.0.1:35418_solr) [n:127.0.0.1:35418_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 632450 INFO (zkCallback-421-thread-2-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 632450 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 632450 INFO (zkCallback-429-thread-1-processing-n:127.0.0.1:34789_solr) [n:127.0.0.1:34789_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 633357 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 633364 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.s.IndexSchema [addReplicaColl_shard1_replica3] Schema name=minimal
[junit4] 2> 633367 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 633368 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard1_replica3' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 633369 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 633369 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.c.SolrCore [[addReplicaColl_shard1_replica3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node3/addReplicaColl_shard1_replica3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-001/node3/./addReplicaColl_shard1_replica3/data/]
[junit4] 2> 633411 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 633411 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 633412 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 633412 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 633413 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.s.SolrIndexSearcher Opening [Searcher@6ad4a027[addReplicaColl_shard1_replica3] main]
[junit4] 2> 633414 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 633414 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 633414 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 633415 INFO (searcherExecutor-1467-thread-1-processing-n:127.0.0.1:34789_solr x:addReplicaColl_shard1_replica3 s:shard1 c:addReplicaColl) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.c.SolrCore [addReplicaColl_shard1_replica3] Registered new searcher Searcher@6ad4a027[addReplicaColl_shard1_replica3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 633416 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576406376078376960
[junit4] 2> 633420 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.c.ZkController Core needs to recover:addReplicaColl_shard1_replica3
[junit4] 2> 633421 INFO (qtp186477155-3008) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 x:addReplicaColl_shard1_replica3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&name=addReplicaColl_shard1_replica3&action=CREATE&collection=addReplicaColl&shard=shard1&wt=javabin&version=2} status=0 QTime=1073
[junit4] 2> 633421 INFO (updateExecutor-420-thread-1-processing-n:127.0.0.1:34789_solr x:addReplicaColl_shard1_replica3 s:shard1 c:addReplicaColl) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica3] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 633421 INFO (recoveryExecutor-422-thread-1-processing-n:127.0.0.1:34789_solr x:addReplicaColl_shard1_replica3 s:shard1 c:addReplicaColl r:core_node5) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica3] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 633423 INFO (recoveryExecutor-422-thread-1-processing-n:127.0.0.1:34789_solr x:addReplicaColl_shard1_replica3 s:shard1 c:addReplicaColl r:core_node5) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica3] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 633423 INFO (recoveryExecutor-422-thread-1-processing-n:127.0.0.1:34789_solr x:addReplicaColl_shard1_replica3 s:shard1 c:addReplicaColl r:core_node5) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica3] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[addReplicaColl_shard1_replica3]
[junit4] 2> 633423 INFO (recoveryExecutor-422-thread-1-processing-n:127.0.0.1:34789_solr x:addReplicaColl_shard1_replica3 s:shard1 c:addReplicaColl r:core_node5) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica3] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 633423 INFO (recoveryExecutor-422-thread-1-processing-n:127.0.0.1:34789_solr x:addReplicaColl_shard1_replica3 s:shard1 c:addReplicaColl r:core_node5) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica3] o.a.s.c.RecoveryStrategy Publishing state of core [addReplicaColl_shard1_replica3] as recovering, leader is [https://127.0.0.1:55021/solr/addReplicaColl_shard1_replica2/] and I am [https://127.0.0.1:34789/solr/addReplicaColl_shard1_replica3/]
[junit4] 2> 633423 INFO (qtp1377695483-3025) [n:127.0.0.1:55021_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:34789_solr&action=ADDREPLICA&collection=addReplicaColl&shard=shard1&wt=javabin&version=2} status=0 QTime=1085
[junit4] 2> 633428 INFO (recoveryExecutor-422-thread-1-processing-n:127.0.0.1:34789_solr x:addReplicaColl_shard1_replica3 s:shard1 c:addReplicaColl r:core_node5) [n:127.0.0.1:34789_solr c:addReplicaColl s:shard1 r:core_node5 x:addReplicaColl_shard1_replica3] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:55021/solr]; [WaitForState: action=PREPRECOVERY&core=addReplicaColl_shard1_replica2&nodeName=127.0.0.1:34789_solr&coreNodeName=core_node5&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 633428 INFO (qtp1377695483-3020) [n:127.0.0.1:55021_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params action=ADDREPLICA&collection=addReplicaColl&shard=shard2&wt=javabin&version=2&property.instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-002 and sendToOCPQueue=true
[junit4] 2> 633432 INFO (OverseerThreadFactory-1438-thread-3-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.AddReplicaCmd addReplica() : {
[junit4] 2> "collection":"addReplicaColl",
[junit4] 2> "shard":"shard2",
[junit4] 2> "property.instanceDir":"/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-002",
[junit4] 2> "operation":"addreplica"}
[junit4] 2> 633432 INFO (OverseerThreadFactory-1438-thread-3-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:35418_solr for creating new replica
[junit4] 2> 633433 INFO (OverseerCollectionConfigSetProcessor-98525398125903879-127.0.0.1:40278_solr-n_0000000000) [n:127.0.0.1:40278_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 633438 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf&name=addReplicaColl_shard2_replica3&action=CREATE&collection=addReplicaColl&shard=shard2&wt=javabin&version=2&property.instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-002
[junit4] 2> 633452 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node5, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true
[junit4] 2> 633453 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr ] o.a.s.h.a.PrepRecoveryOp Will wait a max of 183 seconds to see addReplicaColl_shard1_replica2 (shard1 of addReplicaColl) have state: recovering
[junit4] 2> 633453 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=addReplicaColl, shard=shard1, thisCore=addReplicaColl_shard1_replica2, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:34789_solr, coreNodeName=core_node5, onlyIfActiveCheckResult=false, nodeProps: core_node5:{"core":"addReplicaColl_shard1_replica3","base_url":"https://127.0.0.1:34789/solr","node_name":"127.0.0.1:34789_solr","state":"down"}
[junit4] 2> 633541 INFO (zkCallback-431-thread-2-processing-n:127.0.0.1:35418_solr) [n:127.0.0.1:35418_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 633541 INFO (zkCallback-429-thread-1-processing-n:127.0.0.1:34789_solr) [n:127.0.0.1:34789_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 633541 INFO (zkCallback-424-thread-1-processing-n:127.0.0.1:55021_solr) [n:127.0.0.1:55021_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 633542 INFO (zkCallback-421-thread-2-processing-n:127.0.0.1:40278_solr) [n:127.0.0.1:40278_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addReplicaColl/state.json] for collection [addReplicaColl] has occurred - updating... (live nodes size: [4])
[junit4] 2> 634475 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=addReplicaColl, shard=shard1, thisCore=addReplicaColl_shard1_replica2, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:34789_solr, coreNodeName=core_node5, onlyIfActiveCheckResult=false, nodeProps: core_node5:{"core":"addReplicaColl_shard1_replica3","base_url":"https://127.0.0.1:34789/solr","node_name":"127.0.0.1:34789_solr","state":"recovering"}
[junit4] 2> 634475 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node5, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 634475 INFO (qtp1377695483-3039) [n:127.0.0.1:55021_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:34789_solr&onlyIfLeaderActive=true&core=addReplicaColl_shard1_replica2&coreNodeName=core_node5&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1022
[junit4] 2> 634480 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 634486 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.s.IndexSchema [addReplicaColl_shard2_replica3] Schema name=minimal
[junit4] 2> 634488 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 634489 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.c.CoreContainer Creating SolrCore 'addReplicaColl_shard2_replica3' using configuration from collection addReplicaColl, trusted=true
[junit4] 2> 634489 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 634489 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.c.SolrCore [[addReplicaColl_shard2_replica3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-002], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-002/data/]
[junit4] 2> 634528 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 634528 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 634529 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 634529 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 634530 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.s.SolrIndexSearcher Opening [Searcher@42ffdbc2[addReplicaColl_shard2_replica3] main]
[junit4] 2> 634531 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 634531 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 634531 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 634531 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576406377247539200
[junit4] 2> 634534 INFO (searcherExecutor-1472-thread-1-processing-n:127.0.0.1:35418_solr x:addReplicaColl_shard2_replica3 s:shard2 c:addReplicaColl) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.c.SolrCore [addReplicaColl_shard2_replica3] Registered new searcher Searcher@42ffdbc2[addReplicaColl_shard2_replica3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 634537 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.c.ZkController Core needs to recover:addReplicaColl_shard2_replica3
[junit4] 2> 634537 INFO (updateExecutor-426-thread-1-processing-n:127.0.0.1:35418_solr x:addReplicaColl_shard2_replica3 s:shard2 c:addReplicaColl) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 634537 INFO (recoveryExecutor-427-thread-1-processing-n:127.0.0.1:35418_solr x:addReplicaColl_shard2_replica3 s:shard2 c:addReplicaColl r:core_node6) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 634537 INFO (qtp1381576184-3044) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 x:addReplicaColl_shard2_replica3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf&name=addReplicaColl_shard2_replica3&action=CREATE&collection=addReplicaColl&shard=shard2&wt=javabin&version=2&property.instanceDir=/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-core/test/J1/temp/solr.cloud.CollectionsAPIDistributedZkTest_91A83F8C007FE8E3-001/tempDir-002} status=0 QTime=1099
[junit4] 2> 634538 INFO (recoveryExecutor-427-thread-1-processing-n:127.0.0.1:35418_solr x:addReplicaColl_shard2_replica3 s:shard2 c:addReplicaColl r:core_node6) [n:127.0.0.1:35418_solr c:addReplicaColl s:shard2 r:core_node6 x:addReplicaColl_shard2_replica3] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 634538 INFO (recoveryExecutor-427-thread-1-processing-n:127.0.0.1:35418_solr x:addReplicaColl_shard2_replica3 s:shard2 c:addReplicaColl r:c
[...truncated too long message...]
[junit4] 2> 1800130 INFO (coreLoadExecutor-5010-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1800130 INFO (searcherExecutor-5011-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 1800130 INFO (searcherExecutor-5011-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 1800130 INFO (searcherExecutor-5011-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 1800130 INFO (searcherExecutor-5011-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 1800130 INFO (searcherExecutor-5011-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 1800130 INFO (searcherExecutor-5011-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 1800131 INFO (searcherExecutor-5011-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 1800131 INFO (searcherExecutor-5011-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 1800131 INFO (searcherExecutor-5011-thread-1-processing-x:collection1) [ x:collection1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 1800131 INFO (searcherExecutor-5011-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@407f7f50[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1800131 INFO (coreLoadExecutor-5010-thread-1) [ x:collection1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576407599467724800
[junit4] 2> 1800138 INFO (SUITE-TestJmxIntegration-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.SolrTestCaseJ4 ####initCore end
[junit4] 2> 1800138 INFO (SUITE-TestJmxIntegration-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 1800138 INFO (SUITE-TestJmxIntegration-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=858597635
[junit4] 2> 1800138 INFO (SUITE-TestJmxIntegration-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.node
[junit4] 2> 1800138 INFO (SUITE-TestJmxIntegration-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@436938b3: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / null
[junit4] 2> 1800138 INFO (SUITE-TestJmxIntegration-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jvm
[junit4] 2> 1800138 INFO (SUITE-TestJmxIntegration-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3dc42214: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / null
[junit4] 2> 1800139 INFO (SUITE-TestJmxIntegration-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.jetty
[junit4] 2> 1800139 INFO (SUITE-TestJmxIntegration-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@14772ba9: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / null
[junit4] 2> 1800139 INFO (SUITE-TestJmxIntegration-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1800139 INFO (coreCloseExecutor-5016-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1dac4b77
[junit4] 2> 1800140 INFO (coreCloseExecutor-5016-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.collection1
[junit4] 2> 1800140 INFO (coreCloseExecutor-5016-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@207a51f8: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / null
[junit4] 2> 1800142 INFO (SUITE-TestJmxIntegration-seed#[91A83F8C007FE8E3]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=134, maxMBSortInHeap=6.258878072195851, sim=RandomSimilarity(queryNorm=false,coord=no): {}, locale=sq-AL, timezone=Pacific/Fiji
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=194329872,total=531103744
[junit4] 2> NOTE: All tests run in this JVM: [DistributedDebugComponentTest, CollectionsAPISolrJTest, SolrJmxReporterCloudTest, TestTrieFacet, CursorMarkTest, TestElisionMultitermQuery, SliceStateTest, TestDistributedStatsComponentCardinality, FileUtilsTest, ConvertedLegacyTest, TestFieldCacheSort, HdfsThreadLeakTest, TestIntervalFaceting, TestSuggestSpellingConverter, TestLRUCache, TestGraphMLResponseWriter, DeleteInactiveReplicaTest, ClassificationUpdateProcessorFactoryTest, TestNumericTerms64, TestInPlaceUpdatesStandalone, IndexSchemaTest, TestConfigSets, EchoParamsTest, TestFiltering, TestConfigSetsAPIExclusivity, TestSolrDynamicMBean, PreAnalyzedFieldManagedSchemaCloudTest, SaslZkACLProviderTest, TestWordDelimiterFilterFactory, DirectSolrSpellCheckerTest, TestUseDocValuesAsStored2, SystemInfoHandlerTest, TestTolerantUpdateProcessorCloud, ZkControllerTest, GraphQueryTest, HdfsBasicDistributedZkTest, TestFieldCache, IndexSchemaRuntimeFieldTest, TestPerFieldSimilarityWithDefaultOverride, SampleTest, DocValuesTest, TestCharFilters, TestSearchPerf, TestSimpleTrackingShardHandler, DistributedTermsComponentTest, DistributedIntervalFacetingTest, SolrIndexConfigTest, SolrIndexSplitterTest, TestLMDirichletSimilarityFactory, RemoteQueryErrorTest, TestInitParams, TermVectorComponentTest, TestSQLHandlerNonCloud, TestFuzzyAnalyzedSuggestions, HdfsDirectoryFactoryTest, TestSolrIndexConfig, TestManagedSchemaAPI, SpellCheckCollatorWithCollapseTest, TestCloudDeleteByQuery, TermsComponentTest, SmileWriterTest, DistanceFunctionTest, TestCollationFieldDocValues, CollectionsAPIDistributedZkTest, MetricsHandlerTest, TestLRUStatsCache, SpellCheckCollatorTest, TestLockTree, OutOfBoxZkACLAndCredentialsProvidersTest, TestSubQueryTransformerDistrib, BlockJoinFacetDistribTest, AssignTest, HdfsDirectoryTest, ZkSolrClientTest, TestOrdValues, TestFieldCacheWithThreads, DirectSolrConnectionTest, TestSurroundQueryParser, ForceLeaderTest, TestDistributedGrouping, TestTrackingShardHandlerFactory, TestStressCloudBlindAtomicUpdates, TestBM25SimilarityFactory, TestSchemaManager, TestDynamicLoading, TestLeaderElectionZkExpiry, AnalysisErrorHandlingTest, TestSolrCoreSnapshots, TestSchemaSimilarityResource, RankQueryTest, TestFieldTypeResource, TestSystemIdResolver, SharedFSAutoReplicaFailoverUtilsTest, URLClassifyProcessorTest, TestRestManager, HttpSolrCallGetCoreTest, CollectionStateFormat2Test, OverseerModifyCollectionTest, CdcrRequestHandlerTest, TestExceedMaxTermLength, TestReqParamsAPI, TestSegmentSorting, TestDeleteCollectionOnDownNodes, SortByFunctionTest, TestCloudPivotFacet, SolrMetricsIntegrationTest, TestQueryTypes, TestSolrFieldCacheMBean, HdfsCollectionsAPIDistributedZkTest, TestSolrCloudWithDelegationTokens, TestJsonFacetRefinement, BooleanFieldTest, UpdateRequestProcessorFactoryTest, CopyFieldTest, TemplateUpdateProcessorTest, ZkStateReaderTest, TestRuleBasedAuthorizationPlugin, TestRTGBase, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, SyncSliceTest, OpenCloseCoreStressTest, OverseerTest, LeaderElectionIntegrationTest, ShardRoutingTest, RecoveryZkTest, ClusterStateUpdateTest, TestRandomDVFaceting, TestHashPartitioner, TestStressReorder, TestReload, TestRangeQuery, SpellCheckComponentTest, TestGroupingSearch, TestSort, TestLazyCores, SimplePostToolTest, TestCoreDiscovery, SolrRequestParserTest, TestStressLucene, TestCSVLoader, NoCacheHeaderTest, SchemaVersionSpecificBehaviorTest, QueryEqualityTest, DefaultValueUpdateProcessorTest, TestRemoteStreaming, DocValuesMultiTest, SolrInfoMBeanTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestQueryUtils, PrimitiveFieldTypeTest, DocumentBuilderTest, RequiredFieldsTest, LoggingHandlerTest, TestJmxIntegration]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestJmxIntegration -Dtests.seed=91A83F8C007FE8E3 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=sq-AL -Dtests.timezone=Pacific/Fiji -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestJmxIntegration (suite) <<<
[junit4] > Throwable #1: java.lang.AssertionError: No JMX server found by monitor map
[junit4] > at __randomizedtesting.SeedInfo.seed([91A83F8C007FE8E3]:0)
[junit4] > at org.apache.solr.core.TestJmxIntegration.beforeClass(TestJmxIntegration.java:75)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] Completed [437/713 (3!)] on J1 in 0.98s, 0 tests, 1 failure <<< FAILURES!
[...truncated 44315 lines...]
[JENKINS] Lucene-Solr-Tests-6.6 - Build # 34 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-6.6/34/
1 tests failed.
FAILED: org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure
Error Message:
Did not see a fully active cluster after 30 seconds
Stack Trace:
java.lang.AssertionError: Did not see a fully active cluster after 30 seconds
at __randomizedtesting.SeedInfo.seed([ED764CE729C1E289:6540EEB4F16E0A9B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure(TestCollectionStateWatchers.java:250)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 14091 lines...]
[junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/init-core-data-001
[junit4] 2> 181240 INFO (SUITE-TestCollectionStateWatchers-seed#[ED764CE729C1E289]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields
[junit4] 2> 181240 INFO (SUITE-TestCollectionStateWatchers-seed#[ED764CE729C1E289]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 181253 INFO (SUITE-TestCollectionStateWatchers-seed#[ED764CE729C1E289]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001
[junit4] 2> 181253 INFO (SUITE-TestCollectionStateWatchers-seed#[ED764CE729C1E289]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 181265 INFO (Thread-347) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 181265 INFO (Thread-347) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 181317 ERROR (Thread-347) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 181365 INFO (SUITE-TestCollectionStateWatchers-seed#[ED764CE729C1E289]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:56893
[junit4] 2> 181425 INFO (jetty-launcher-549-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 181434 INFO (jetty-launcher-549-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 181436 INFO (jetty-launcher-549-thread-3) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 181443 INFO (jetty-launcher-549-thread-4) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 181593 INFO (jetty-launcher-549-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@49bbcbb8{/solr,null,AVAILABLE}
[junit4] 2> 181595 INFO (jetty-launcher-549-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6f988f36{SSL,[ssl, http/1.1]}{127.0.0.1:43452}
[junit4] 2> 181595 INFO (jetty-launcher-549-thread-3) [ ] o.e.j.s.Server Started @189267ms
[junit4] 2> 181595 INFO (jetty-launcher-549-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43452}
[junit4] 2> 181595 ERROR (jetty-launcher-549-thread-3) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 181595 INFO (jetty-launcher-549-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 181595 INFO (jetty-launcher-549-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 181595 INFO (jetty-launcher-549-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 181595 INFO (jetty-launcher-549-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-21T08:30:44.697Z
[junit4] 2> 181684 INFO (jetty-launcher-549-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 181729 INFO (jetty-launcher-549-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5cbc4652{/solr,null,AVAILABLE}
[junit4] 2> 181730 INFO (jetty-launcher-549-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3a8b8def{SSL,[ssl, http/1.1]}{127.0.0.1:52294}
[junit4] 2> 181730 INFO (jetty-launcher-549-thread-4) [ ] o.e.j.s.Server Started @189402ms
[junit4] 2> 181730 INFO (jetty-launcher-549-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52294}
[junit4] 2> 181730 ERROR (jetty-launcher-549-thread-4) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 181730 INFO (jetty-launcher-549-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 181730 INFO (jetty-launcher-549-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 181730 INFO (jetty-launcher-549-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 181730 INFO (jetty-launcher-549-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-21T08:30:44.832Z
[junit4] 2> 181750 INFO (jetty-launcher-549-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@16b85cdb{/solr,null,AVAILABLE}
[junit4] 2> 181751 INFO (jetty-launcher-549-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2efe6212{SSL,[ssl, http/1.1]}{127.0.0.1:54513}
[junit4] 2> 181751 INFO (jetty-launcher-549-thread-1) [ ] o.e.j.s.Server Started @189423ms
[junit4] 2> 181751 INFO (jetty-launcher-549-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54513}
[junit4] 2> 181751 ERROR (jetty-launcher-549-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 181751 INFO (jetty-launcher-549-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 181751 INFO (jetty-launcher-549-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 181751 INFO (jetty-launcher-549-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 181751 INFO (jetty-launcher-549-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-21T08:30:44.853Z
[junit4] 2> 181762 INFO (jetty-launcher-549-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@570965cc{/solr,null,AVAILABLE}
[junit4] 2> 181763 INFO (jetty-launcher-549-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4c578abc{SSL,[ssl, http/1.1]}{127.0.0.1:53851}
[junit4] 2> 181763 INFO (jetty-launcher-549-thread-2) [ ] o.e.j.s.Server Started @189435ms
[junit4] 2> 181763 INFO (jetty-launcher-549-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53851}
[junit4] 2> 181763 ERROR (jetty-launcher-549-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 181763 INFO (jetty-launcher-549-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 6.6.1
[junit4] 2> 181763 INFO (jetty-launcher-549-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 181763 INFO (jetty-launcher-549-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 181763 INFO (jetty-launcher-549-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-21T08:30:44.865Z
[junit4] 2> 181764 INFO (jetty-launcher-549-thread-3) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 181765 INFO (jetty-launcher-549-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56893/solr
[junit4] 2> 181777 INFO (jetty-launcher-549-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 181791 INFO (jetty-launcher-549-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 181795 INFO (jetty-launcher-549-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 181798 INFO (jetty-launcher-549-thread-1) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 181798 INFO (jetty-launcher-549-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56893/solr
[junit4] 2> 181800 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e03ea5c970002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 181809 INFO (jetty-launcher-549-thread-4) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 181822 INFO (jetty-launcher-549-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56893/solr
[junit4] 2> 181825 INFO (jetty-launcher-549-thread-2) [ ] o.a.s.u.UpdateShardHandler Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=true
[junit4] 2> 181832 INFO (jetty-launcher-549-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56893/solr
[junit4] 2> 181911 INFO (jetty-launcher-549-thread-3) [n:127.0.0.1:43452_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 181925 INFO (jetty-launcher-549-thread-1) [n:127.0.0.1:54513_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 181936 INFO (jetty-launcher-549-thread-3) [n:127.0.0.1:43452_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43452_solr
[junit4] 2> 181939 INFO (jetty-launcher-549-thread-1) [n:127.0.0.1:54513_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54513_solr
[junit4] 2> 181939 INFO (jetty-launcher-549-thread-3) [n:127.0.0.1:43452_solr ] o.a.s.c.Overseer Overseer (id=98520546959360006-127.0.0.1:43452_solr-n_0000000000) starting
[junit4] 2> 182007 INFO (jetty-launcher-549-thread-4) [n:127.0.0.1:52294_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 182010 INFO (jetty-launcher-549-thread-4) [n:127.0.0.1:52294_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 182014 INFO (jetty-launcher-549-thread-4) [n:127.0.0.1:52294_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52294_solr
[junit4] 2> 182023 INFO (zkCallback-569-thread-1-processing-n:127.0.0.1:54513_solr) [n:127.0.0.1:54513_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 182029 INFO (jetty-launcher-549-thread-2) [n:127.0.0.1:53851_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 182030 INFO (jetty-launcher-549-thread-2) [n:127.0.0.1:53851_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 182034 INFO (jetty-launcher-549-thread-2) [n:127.0.0.1:53851_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53851_solr
[junit4] 2> 182036 INFO (zkCallback-569-thread-1-processing-n:127.0.0.1:54513_solr) [n:127.0.0.1:54513_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 182150 INFO (jetty-launcher-549-thread-3) [n:127.0.0.1:43452_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43452_solr
[junit4] 2> 182151 INFO (zkCallback-569-thread-1-processing-n:127.0.0.1:54513_solr) [n:127.0.0.1:54513_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 182213 INFO (zkCallback-571-thread-2-processing-n:127.0.0.1:52294_solr) [n:127.0.0.1:52294_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
[junit4] 2> 182226 INFO (zkCallback-559-thread-1-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 182226 INFO (OverseerStateUpdate-98520546959360006-127.0.0.1:43452_solr-n_0000000000) [n:127.0.0.1:43452_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 182305 INFO (zkCallback-572-thread-1-processing-n:127.0.0.1:53851_solr) [n:127.0.0.1:53851_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 182496 INFO (jetty-launcher-549-thread-3) [n:127.0.0.1:43452_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node3/.
[junit4] 2> 182506 INFO (jetty-launcher-549-thread-1) [n:127.0.0.1:54513_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node1/.
[junit4] 2> 182525 INFO (jetty-launcher-549-thread-2) [n:127.0.0.1:53851_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node2/.
[junit4] 2> 182636 INFO (jetty-launcher-549-thread-4) [n:127.0.0.1:52294_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node4/.
[junit4] 2> 182739 INFO (SUITE-TestCollectionStateWatchers-seed#[ED764CE729C1E289]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 182739 INFO (SUITE-TestCollectionStateWatchers-seed#[ED764CE729C1E289]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56893/solr ready
[junit4] 2> 182834 INFO (TEST-TestCollectionStateWatchers.testCanWaitForNonexistantCollection-seed#[ED764CE729C1E289]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCanWaitForNonexistantCollection
[junit4] 2> 183082 INFO (qtp1924484925-1861) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&replicationFactor=1&collection.configName=config&name=delayed&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 183085 INFO (qtp1924484925-1861) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&replicationFactor=1&collection.configName=config&name=delayed&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 183091 INFO (qtp1924484925-1861) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 183092 INFO (qtp1924484925-1861) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 183093 INFO (OverseerThreadFactory-745-thread-1-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.CreateCollectionCmd Create collection delayed
[junit4] 2> 183116 INFO (zkCallback-579-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeCreated path:/collections/delayed/state.json] for collection [delayed] has occurred - updating... (live nodes size: [4])
[junit4] 2> 183284 INFO (qtp1019341583-1850) [n:127.0.0.1:43452_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719&qt=/admin/cores&collection.configName=config&newCollection=true&name=delayed_shard1_replica1&action=CREATE&numShards=1&collection=delayed&shard=shard1&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 183285 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr ] o.a.s.h.a.CoreAdminOperation core create command async=0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719&qt=/admin/cores&collection.configName=config&newCollection=true&name=delayed_shard1_replica1&action=CREATE&numShards=1&collection=delayed&shard=shard1&wt=javabin&version=2
[junit4] 2> 183285 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 183313 INFO (qtp1019341583-1851) [n:127.0.0.1:43452_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719
[junit4] 2> 183313 INFO (qtp1019341583-1851) [n:127.0.0.1:43452_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 183585 INFO (zkCallback-559-thread-2-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delayed/state.json] for collection [delayed] has occurred - updating... (live nodes size: [4])
[junit4] 2> 183586 INFO (zkCallback-579-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delayed/state.json] for collection [delayed] has occurred - updating... (live nodes size: [4])
[junit4] 2> 184099 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 184101 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 184314 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 184316 INFO (qtp1019341583-1852) [n:127.0.0.1:43452_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719
[junit4] 2> 184316 INFO (qtp1019341583-1852) [n:127.0.0.1:43452_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 184334 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.s.IndexSchema [delayed_shard1_replica1] Schema name=test
[junit4] 2> 184417 WARN (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 184418 WARN (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 184418 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 184418 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'delayed_shard1_replica1' using configuration from collection delayed, trusted=true
[junit4] 2> 184419 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 184419 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.SolrCore [[delayed_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node3/delayed_shard1_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node3/./delayed_shard1_replica1/data/]
[junit4] 2> 184446 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 184446 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 184447 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 184447 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 184448 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@323dc9b7[delayed_shard1_replica1] main]
[junit4] 2> 184449 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 184449 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 184449 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 184449 INFO (searcherExecutor-754-thread-1-processing-x:delayed_shard1_replica1 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE n:127.0.0.1:43452_solr s:shard1 c:delayed) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.SolrCore [delayed_shard1_replica1] Registered new searcher Searcher@323dc9b7[delayed_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 184449 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576328754680037376
[junit4] 2> 184456 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 184456 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 184456 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:43452/solr/delayed_shard1_replica1/
[junit4] 2> 184456 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 184456 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:43452/solr/delayed_shard1_replica1/ has no replicas
[junit4] 2> 184456 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 184460 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:43452/solr/delayed_shard1_replica1/ shard1
[junit4] 2> 184564 INFO (zkCallback-579-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delayed/state.json] for collection [delayed] has occurred - updating... (live nodes size: [4])
[junit4] 2> 184564 INFO (zkCallback-559-thread-2-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delayed/state.json] for collection [delayed] has occurred - updating... (live nodes size: [4])
[junit4] 2> 184611 INFO (parallelCoreAdminExecutor-746-thread-1-processing-n:127.0.0.1:43452_solr 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719 CREATE) [n:127.0.0.1:43452_solr c:delayed s:shard1 x:delayed_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 184715 INFO (zkCallback-579-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delayed/state.json] for collection [delayed] has occurred - updating... (live nodes size: [4])
[junit4] 2> 184718 INFO (zkCallback-559-thread-2-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/delayed/state.json] for collection [delayed] has occurred - updating... (live nodes size: [4])
[junit4] 2> 185104 INFO (qtp1924484925-1863) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 185110 INFO (qtp1924484925-1863) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 185331 INFO (qtp1019341583-1845) [n:127.0.0.1:43452_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719
[junit4] 2> 185331 INFO (qtp1019341583-1845) [n:127.0.0.1:43452_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb3556178158978719&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 185339 INFO (OverseerThreadFactory-745-thread-1-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 186113 INFO (qtp1924484925-1879) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 186116 INFO (qtp1924484925-1879) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 186120 INFO (qtp1924484925-1857) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 186121 INFO (qtp1924484925-1857) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=0bab450f-7976-42e7-a6a3-e2ad43ed03bb&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 186122 INFO (TEST-TestCollectionStateWatchers.testCanWaitForNonexistantCollection-seed#[ED764CE729C1E289]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCanWaitForNonexistantCollection
[junit4] 2> 186151 INFO (TEST-TestCollectionStateWatchers.testStateWatcherChecksCurrentStateOnRegister-seed#[ED764CE729C1E289]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testStateWatcherChecksCurrentStateOnRegister
[junit4] 2> 186157 INFO (qtp1924484925-1860) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=3a37a039-c087-4bbd-8e00-1f39a66c0f54&replicationFactor=1&collection.configName=config&name=currentstate&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 186159 INFO (qtp1924484925-1860) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=3a37a039-c087-4bbd-8e00-1f39a66c0f54&replicationFactor=1&collection.configName=config&name=currentstate&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 186160 INFO (OverseerThreadFactory-745-thread-2-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.CreateCollectionCmd Create collection currentstate
[junit4] 2> 186161 INFO (qtp1924484925-1861) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f54&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 186161 INFO (OverseerCollectionConfigSetProcessor-98520546959360006-127.0.0.1:43452_solr-n_0000000000) [n:127.0.0.1:43452_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 186164 INFO (qtp1924484925-1861) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f54&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 186376 INFO (qtp1924484925-1858) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737&qt=/admin/cores&collection.configName=config&newCollection=true&name=currentstate_shard1_replica1&action=CREATE&numShards=1&collection=currentstate&shard=shard1&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 186377 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CoreAdminOperation core create command async=3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737&qt=/admin/cores&collection.configName=config&newCollection=true&name=currentstate_shard1_replica1&action=CREATE&numShards=1&collection=currentstate&shard=shard1&wt=javabin&version=2
[junit4] 2> 186378 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 186379 INFO (qtp1924484925-1879) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737
[junit4] 2> 186379 INFO (qtp1924484925-1879) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 186485 INFO (zkCallback-572-thread-1-processing-n:127.0.0.1:53851_solr) [n:127.0.0.1:53851_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/currentstate/state.json] for collection [currentstate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 187167 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f54&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 187168 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f54&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 187382 INFO (qtp1924484925-1857) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737
[junit4] 2> 187382 INFO (qtp1924484925-1857) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 187395 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 187403 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.s.IndexSchema [currentstate_shard1_replica1] Schema name=test
[junit4] 2> 187470 WARN (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 187470 WARN (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 187471 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 187471 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'currentstate_shard1_replica1' using configuration from collection currentstate, trusted=true
[junit4] 2> 187471 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 187471 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.SolrCore [[currentstate_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node2/currentstate_shard1_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node2/./currentstate_shard1_replica1/data/]
[junit4] 2> 187503 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 187503 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 187504 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 187504 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 187505 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3fe5f64b[currentstate_shard1_replica1] main]
[junit4] 2> 187507 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 187508 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 187508 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 187509 INFO (searcherExecutor-759-thread-1-processing-x:currentstate_shard1_replica1 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE n:127.0.0.1:53851_solr s:shard1 c:currentstate) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.SolrCore [currentstate_shard1_replica1] Registered new searcher Searcher@3fe5f64b[currentstate_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 187509 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576328757888679936
[junit4] 2> 187518 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 187518 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 187518 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:53851/solr/currentstate_shard1_replica1/
[junit4] 2> 187518 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 187518 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:53851/solr/currentstate_shard1_replica1/ has no replicas
[junit4] 2> 187518 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 187522 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:53851/solr/currentstate_shard1_replica1/ shard1
[junit4] 2> 187625 INFO (zkCallback-572-thread-1-processing-n:127.0.0.1:53851_solr) [n:127.0.0.1:53851_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/currentstate/state.json] for collection [currentstate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 187675 INFO (parallelCoreAdminExecutor-750-thread-1-processing-n:127.0.0.1:53851_solr 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737 CREATE) [n:127.0.0.1:53851_solr c:currentstate s:shard1 x:currentstate_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 187779 INFO (zkCallback-572-thread-1-processing-n:127.0.0.1:53851_solr) [n:127.0.0.1:53851_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/currentstate/state.json] for collection [currentstate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 188171 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f54&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 188172 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f54&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 188386 INFO (qtp1924484925-1857) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : 3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737
[junit4] 2> 188386 INFO (qtp1924484925-1857) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f543556181317324737&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 188390 INFO (OverseerThreadFactory-745-thread-2-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000001 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 189175 INFO (qtp1924484925-1863) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f54&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 189177 INFO (qtp1924484925-1863) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f54&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 189180 INFO (qtp1924484925-1879) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f54&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 189180 INFO (qtp1924484925-1879) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=3a37a039-c087-4bbd-8e00-1f39a66c0f54&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 189186 INFO (TEST-TestCollectionStateWatchers.testStateWatcherChecksCurrentStateOnRegister-seed#[ED764CE729C1E289]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testStateWatcherChecksCurrentStateOnRegister
[junit4] 2> 189211 INFO (TEST-TestCollectionStateWatchers.testWaitForStateChecksCurrentState-seed#[ED764CE729C1E289]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testWaitForStateChecksCurrentState
[junit4] 2> 189216 INFO (qtp1924484925-1860) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&replicationFactor=1&collection.configName=config&name=waitforstate&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 189218 INFO (qtp1924484925-1860) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&replicationFactor=1&collection.configName=config&name=waitforstate&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 189222 INFO (qtp1924484925-1861) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 189234 INFO (qtp1924484925-1861) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=11
[junit4] 2> 189235 INFO (OverseerThreadFactory-745-thread-3-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.CreateCollectionCmd Create collection waitforstate
[junit4] 2> 189236 INFO (OverseerCollectionConfigSetProcessor-98520546959360006-127.0.0.1:43452_solr-n_0000000000) [n:127.0.0.1:43452_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000001 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 189547 INFO (qtp1911332881-1883) [n:127.0.0.1:54513_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797&qt=/admin/cores&collection.configName=config&newCollection=true&name=waitforstate_shard1_replica1&action=CREATE&numShards=1&collection=waitforstate&shard=shard1&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 189547 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr ] o.a.s.h.a.CoreAdminOperation core create command async=b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797&qt=/admin/cores&collection.configName=config&newCollection=true&name=waitforstate_shard1_replica1&action=CREATE&numShards=1&collection=waitforstate&shard=shard1&wt=javabin&version=2
[junit4] 2> 189547 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 189550 INFO (qtp1911332881-1884) [n:127.0.0.1:54513_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797
[junit4] 2> 189550 INFO (qtp1911332881-1884) [n:127.0.0.1:54513_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 189654 INFO (zkCallback-569-thread-1-processing-n:127.0.0.1:54513_solr) [n:127.0.0.1:54513_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/waitforstate/state.json] for collection [waitforstate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 190236 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 190237 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 190556 INFO (qtp1911332881-1885) [n:127.0.0.1:54513_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797
[junit4] 2> 190556 INFO (qtp1911332881-1885) [n:127.0.0.1:54513_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 190569 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 190583 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.s.IndexSchema [waitforstate_shard1_replica1] Schema name=test
[junit4] 2> 190727 WARN (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 190727 WARN (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 190732 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 190732 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.CoreContainer Creating SolrCore 'waitforstate_shard1_replica1' using configuration from collection waitforstate, trusted=true
[junit4] 2> 190732 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 190732 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.SolrCore [[waitforstate_shard1_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node1/waitforstate_shard1_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node1/./waitforstate_shard1_replica1/data/]
[junit4] 2> 190770 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 190770 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 190770 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 190770 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 190772 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@47af90e5[waitforstate_shard1_replica1] main]
[junit4] 2> 190785 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 190785 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 190785 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 190786 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1576328761324863488
[junit4] 2> 190792 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 190792 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 190792 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:54513/solr/waitforstate_shard1_replica1/
[junit4] 2> 190792 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 190792 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.SyncStrategy https://127.0.0.1:54513/solr/waitforstate_shard1_replica1/ has no replicas
[junit4] 2> 190792 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 190808 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:54513/solr/waitforstate_shard1_replica1/ shard1
[junit4] 2> 190808 INFO (searcherExecutor-764-thread-1-processing-x:waitforstate_shard1_replica1 b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE n:127.0.0.1:54513_solr s:shard1 c:waitforstate) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.SolrCore [waitforstate_shard1_replica1] Registered new searcher Searcher@47af90e5[waitforstate_shard1_replica1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 190900 INFO (zkCallback-569-thread-1-processing-n:127.0.0.1:54513_solr) [n:127.0.0.1:54513_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/waitforstate/state.json] for collection [waitforstate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 190909 INFO (parallelCoreAdminExecutor-748-thread-1-processing-n:127.0.0.1:54513_solr b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797 CREATE) [n:127.0.0.1:54513_solr c:waitforstate s:shard1 x:waitforstate_shard1_replica1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 191019 INFO (zkCallback-569-thread-1-processing-n:127.0.0.1:54513_solr) [n:127.0.0.1:54513_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/waitforstate/state.json] for collection [waitforstate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 191239 INFO (qtp1924484925-1858) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 191240 INFO (qtp1924484925-1858) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 191559 INFO (qtp1911332881-1864) [n:127.0.0.1:54513_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797
[junit4] 2> 191559 INFO (qtp1911332881-1864) [n:127.0.0.1:54513_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d53556184418123797&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 191562 INFO (OverseerThreadFactory-745-thread-3-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 192243 INFO (qtp1924484925-1863) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 192245 INFO (qtp1924484925-1863) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 192252 INFO (qtp1924484925-1863) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletestatus with params requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&action=DELETESTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 192252 INFO (qtp1924484925-1863) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=b9e1bc96-cfda-40f6-8c19-3263fd8da3d5&action=DELETESTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 192258 INFO (TEST-TestCollectionStateWatchers.testWaitForStateChecksCurrentState-seed#[ED764CE729C1E289]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testWaitForStateChecksCurrentState
[junit4] 2> 192283 INFO (TEST-TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure-seed#[ED764CE729C1E289]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testWaitForStateWatcherIsRetainedOnPredicateFailure
[junit4] 2> 192287 INFO (qtp1924484925-1879) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params async=de92a76c-2d50-4fc5-a512-a12b1e47a50b&replicationFactor=1&collection.configName=config&name=falsepredicate&action=CREATE&numShards=4&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 192288 INFO (qtp1924484925-1879) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={async=de92a76c-2d50-4fc5-a512-a12b1e47a50b&replicationFactor=1&collection.configName=config&name=falsepredicate&action=CREATE&numShards=4&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 192292 INFO (qtp1924484925-1860) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=de92a76c-2d50-4fc5-a512-a12b1e47a50b&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 192295 INFO (qtp1924484925-1860) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=de92a76c-2d50-4fc5-a512-a12b1e47a50b&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 192301 INFO (OverseerCollectionConfigSetProcessor-98520546959360006-127.0.0.1:43452_solr-n_0000000000) [n:127.0.0.1:43452_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 192302 INFO (OverseerThreadFactory-745-thread-4-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.CreateCollectionCmd Create collection falsepredicate
[junit4] 2> 192642 INFO (qtp1924484925-1857) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187581985338&qt=/admin/cores&collection.configName=config&newCollection=true&name=falsepredicate_shard3_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard3&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 192644 INFO (parallelCoreAdminExecutor-750-thread-2-processing-n:127.0.0.1:53851_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187581985338 CREATE) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CoreAdminOperation core create command async=de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187581985338&qt=/admin/cores&collection.configName=config&newCollection=true&name=falsepredicate_shard3_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard3&wt=javabin&version=2
[junit4] 2> 192646 INFO (qtp1019341583-1845) [n:127.0.0.1:43452_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940&qt=/admin/cores&collection.configName=config&newCollection=true&name=falsepredicate_shard4_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard4&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 192647 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr ] o.a.s.h.a.CoreAdminOperation core create command async=de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940&qt=/admin/cores&collection.configName=config&newCollection=true&name=falsepredicate_shard4_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard4&wt=javabin&version=2
[junit4] 2> 192665 INFO (qtp1911332881-1864) [n:127.0.0.1:54513_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187582113299&qt=/admin/cores&collection.configName=config&newCollection=true&name=falsepredicate_shard2_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard2&wt=javabin&version=2} status=0 QTime=21
[junit4] 2> 192677 INFO (parallelCoreAdminExecutor-748-thread-2-processing-n:127.0.0.1:54513_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187582113299 CREATE) [n:127.0.0.1:54513_solr ] o.a.s.h.a.CoreAdminOperation core create command async=de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187582113299&qt=/admin/cores&collection.configName=config&newCollection=true&name=falsepredicate_shard2_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard2&wt=javabin&version=2
[junit4] 2> 192757 INFO (qtp640153030-1871) [n:127.0.0.1:52294_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={async=de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588872924&qt=/admin/cores&collection.configName=config&newCollection=true&name=falsepredicate_shard1_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard1&wt=javabin&version=2} status=0 QTime=26
[junit4] 2> 192758 INFO (parallelCoreAdminExecutor-752-thread-1-processing-n:127.0.0.1:52294_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588872924 CREATE) [n:127.0.0.1:52294_solr ] o.a.s.h.a.CoreAdminOperation core create command async=de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588872924&qt=/admin/cores&collection.configName=config&newCollection=true&name=falsepredicate_shard1_replica1&action=CREATE&numShards=4&collection=falsepredicate&shard=shard1&wt=javabin&version=2
[junit4] 2> 192758 INFO (parallelCoreAdminExecutor-752-thread-1-processing-n:127.0.0.1:52294_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588872924 CREATE) [n:127.0.0.1:52294_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 192762 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187581985338
[junit4] 2> 192763 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187581985338&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 192939 INFO (zkCallback-569-thread-1-processing-n:127.0.0.1:54513_solr) [n:127.0.0.1:54513_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 192939 INFO (zkCallback-572-thread-1-processing-n:127.0.0.1:53851_solr) [n:127.0.0.1:53851_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 192939 INFO (zkCallback-571-thread-1-processing-n:127.0.0.1:52294_solr) [n:127.0.0.1:52294_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 192940 INFO (zkCallback-559-thread-2-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/falsepredicate/state.json] for collection [falsepredicate] has occurred - updating... (live nodes size: [4])
[junit4] 2> 193297 INFO (qtp1924484925-1861) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :requeststatus with params requestid=de92a76c-2d50-4fc5-a512-a12b1e47a50b&action=REQUESTSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 193298 INFO (qtp1924484925-1861) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={requestid=de92a76c-2d50-4fc5-a512-a12b1e47a50b&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 193673 INFO (parallelCoreAdminExecutor-750-thread-2-processing-n:127.0.0.1:53851_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187581985338 CREATE) [n:127.0.0.1:53851_solr c:falsepredicate s:shard3 x:falsepredicate_shard3_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 193673 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 193683 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.s.IndexSchema [falsepredicate_shard4_replica1] Schema name=test
[junit4] 2> 193695 INFO (parallelCoreAdminExecutor-748-thread-2-processing-n:127.0.0.1:54513_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187582113299 CREATE) [n:127.0.0.1:54513_solr c:falsepredicate s:shard2 x:falsepredicate_shard2_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 193744 INFO (parallelCoreAdminExecutor-748-thread-2-processing-n:127.0.0.1:54513_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187582113299 CREATE) [n:127.0.0.1:54513_solr c:falsepredicate s:shard2 x:falsepredicate_shard2_replica1] o.a.s.s.IndexSchema [falsepredicate_shard2_replica1] Schema name=test
[junit4] 2> 193750 INFO (parallelCoreAdminExecutor-750-thread-2-processing-n:127.0.0.1:53851_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187581985338 CREATE) [n:127.0.0.1:53851_solr c:falsepredicate s:shard3 x:falsepredicate_shard3_replica1] o.a.s.s.IndexSchema [falsepredicate_shard3_replica1] Schema name=test
[junit4] 2> 193765 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.h.a.CoreAdminOperation Checking request status for : de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187581985338
[junit4] 2> 193766 INFO (qtp1924484925-1862) [n:127.0.0.1:53851_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&requestid=de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187581985338&action=REQUESTSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 193842 INFO (parallelCoreAdminExecutor-752-thread-1-processing-n:127.0.0.1:52294_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588872924 CREATE) [n:127.0.0.1:52294_solr c:falsepredicate s:shard1 x:falsepredicate_shard1_replica1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 6.6.1
[junit4] 2> 193843 WARN (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 193843 WARN (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 193843 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 193843 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.c.CoreContainer Creating SolrCore 'falsepredicate_shard4_replica1' using configuration from collection falsepredicate, trusted=true
[junit4] 2> 193844 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 193844 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.c.SolrCore [[falsepredicate_shard4_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node3/falsepredicate_shard4_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node3/./falsepredicate_shard4_replica1/data/]
[junit4] 2> 193859 INFO (parallelCoreAdminExecutor-752-thread-1-processing-n:127.0.0.1:52294_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588872924 CREATE) [n:127.0.0.1:52294_solr c:falsepredicate s:shard1 x:falsepredicate_shard1_replica1] o.a.s.s.IndexSchema [falsepredicate_shard1_replica1] Schema name=test
[junit4] 2> 193980 WARN (parallelCoreAdminExecutor-748-thread-2-processing-n:127.0.0.1:54513_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187582113299 CREATE) [n:127.0.0.1:54513_solr c:falsepredicate s:shard2 x:falsepredicate_shard2_replica1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 193981 WARN (parallelCoreAdminExecutor-748-thread-2-processing-n:127.0.0.1:54513_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187582113299 CREATE) [n:127.0.0.1:54513_solr c:falsepredicate s:shard2 x:falsepredicate_shard2_replica1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 193984 INFO (parallelCoreAdminExecutor-748-thread-2-processing-n:127.0.0.1:54513_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187582113299 CREATE) [n:127.0.0.1:54513_solr c:falsepredicate s:shard2 x:falsepredicate_shard2_replica1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 193984 INFO (parallelCoreAdminExecutor-748-thread-2-processing-n:127.0.0.1:54513_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187582113299 CREATE) [n:127.0.0.1:54513_solr c:falsepredicate s:shard2 x:falsepredicate_shard2_replica1] o.a.s.c.CoreContainer Creating SolrCore 'falsepredicate_shard2_replica1' using configuration from collection falsepredicate, trusted=true
[junit4] 2> 193988 INFO (parallelCoreAdminExecutor-748-thread-2-processing-n:127.0.0.1:54513_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187582113299 CREATE) [n:127.0.0.1:54513_solr c:falsepredicate s:shard2 x:falsepredicate_shard2_replica1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 193988 INFO (parallelCoreAdminExecutor-748-thread-2-processing-n:127.0.0.1:54513_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187582113299 CREATE) [n:127.0.0.1:54513_solr c:falsepredicate s:shard2 x:falsepredicate_shard2_replica1] o.a.s.c.SolrCore [[falsepredicate_shard2_replica1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node1/falsepredicate_shard2_replica1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001/tempDir-001/node1/./falsepredicate_shard2_replica1/data/]
[junit4] 2> 194010 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 194010 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 194011 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 194011 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 194012 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ffe694a[falsepredicate_shard4_replica1] main]
[junit4] 2> 194014 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 194016 INFO (parallelCoreAdminExecutor-746-thread-2-processing-n:127.0.0.1:43452_solr de92a76c-2d50-4fc5-a512-a12b1e47a50b3556187588045940 CREATE) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 x:falsepredicate_shard4_replica1] o.a.
[...truncated too long message...]
r... (4) -> (3)
[junit4] 2> 238681 INFO (zkCallback-602-thread-1-processing-n:127.0.0.1:52786_solr) [n:127.0.0.1:52786_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 238681 INFO (zkCallback-594-thread-2-processing-n:127.0.0.1:41606_solr) [n:127.0.0.1:41606_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 238682 INFO (jetty-closer-550-thread-4) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 238690 INFO (coreCloseExecutor-861-thread-1) [n:127.0.0.1:41606_solr c:falsepredicate s:shard1 r:core_node6 x:falsepredicate_shard1_replica1] o.a.s.c.SolrCore [falsepredicate_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@76fd5f2c
[junit4] 2> 238690 INFO (coreCloseExecutor-861-thread-2) [n:127.0.0.1:41606_solr c:stateformat1 s:shard1 r:core_node1 x:stateformat1_shard1_replica1] o.a.s.c.SolrCore [stateformat1_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@56e2d927
[junit4] 2> 238693 INFO (coreCloseExecutor-862-thread-1) [n:127.0.0.1:43452_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.c.SolrCore [delayed_shard1_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@526a09a7
[junit4] 2> 238694 INFO (zkCallback-594-thread-1-processing-n:127.0.0.1:41606_solr) [n:127.0.0.1:41606_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 238694 INFO (zkCallback-559-thread-2-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 238695 INFO (coreCloseExecutor-861-thread-3) [n:127.0.0.1:41606_solr c:testcollection s:shard2 r:core_node1 x:testcollection_shard2_replica1] o.a.s.c.SolrCore [testcollection_shard2_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@14d19a19
[junit4] 2> 238701 INFO (coreCloseExecutor-862-thread-2) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 r:core_node1 x:falsepredicate_shard4_replica1] o.a.s.c.SolrCore [falsepredicate_shard4_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@718cdf01
[junit4] 2> 238702 INFO (coreCloseExecutor-861-thread-1) [n:127.0.0.1:41606_solr c:falsepredicate s:shard1 r:core_node6 x:falsepredicate_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.falsepredicate.shard1.replica1
[junit4] 2> 238709 INFO (coreCloseExecutor-861-thread-2) [n:127.0.0.1:41606_solr c:stateformat1 s:shard1 r:core_node1 x:stateformat1_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.stateformat1.shard1.replica1
[junit4] 2> 238709 INFO (coreCloseExecutor-862-thread-3) [n:127.0.0.1:43452_solr c:testcollection s:shard3 r:core_node3 x:testcollection_shard3_replica1] o.a.s.c.SolrCore [testcollection_shard3_replica1] CLOSING SolrCore org.apache.solr.core.SolrCore@42e70be4
[junit4] 2> 238709 INFO (coreCloseExecutor-862-thread-2) [n:127.0.0.1:43452_solr c:falsepredicate s:shard4 r:core_node1 x:falsepredicate_shard4_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.falsepredicate.shard4.replica1
[junit4] 2> 238710 INFO (coreCloseExecutor-862-thread-1) [n:127.0.0.1:43452_solr c:delayed s:shard1 r:core_node1 x:delayed_shard1_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.delayed.shard1.replica1
[junit4] 2> 238713 INFO (coreCloseExecutor-861-thread-3) [n:127.0.0.1:41606_solr c:testcollection s:shard2 r:core_node1 x:testcollection_shard2_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testcollection.shard2.replica1
[junit4] 2> 238720 INFO (coreCloseExecutor-862-thread-3) [n:127.0.0.1:43452_solr c:testcollection s:shard3 r:core_node3 x:testcollection_shard3_replica1] o.a.s.m.SolrMetricManager Closing metric reporters for: solr.core.testcollection.shard3.replica1
[junit4] 2> 238722 INFO (zkCallback-559-thread-2-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 238724 INFO (jetty-closer-550-thread-2) [ ] o.a.s.c.Overseer Overseer (id=98520546959360006-127.0.0.1:43452_solr-n_0000000000) closing
[junit4] 2> 238725 INFO (OverseerStateUpdate-98520546959360006-127.0.0.1:43452_solr-n_0000000000) [n:127.0.0.1:43452_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:43452_solr
[junit4] 2> 240181 WARN (zkCallback-572-thread-2-processing-n:127.0.0.1:53851_solr) [n:127.0.0.1:53851_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 240181 INFO (jetty-closer-550-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@570965cc{/solr,null,UNAVAILABLE}
[junit4] 2> 240196 WARN (zkCallback-602-thread-1-processing-n:127.0.0.1:52786_solr) [n:127.0.0.1:52786_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 240196 INFO (jetty-closer-550-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@34586f96{/solr,null,UNAVAILABLE}
[junit4] 2> 240223 WARN (zkCallback-594-thread-1-processing-n:127.0.0.1:41606_solr) [n:127.0.0.1:41606_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 240223 INFO (jetty-closer-550-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@93f9931{/solr,null,UNAVAILABLE}
[junit4] 2> 240229 WARN (zkCallback-559-thread-2-processing-n:127.0.0.1:43452_solr) [n:127.0.0.1:43452_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 240230 INFO (jetty-closer-550-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@49bbcbb8{/solr,null,UNAVAILABLE}
[junit4] 2> 240237 ERROR (SUITE-TestCollectionStateWatchers-seed#[ED764CE729C1E289]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 240238 INFO (SUITE-TestCollectionStateWatchers-seed#[ED764CE729C1E289]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56893 56893
[junit4] 2> 240329 INFO (Thread-347) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:56893 56893
[junit4] 2> 240330 WARN (Thread-347) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 7 /solr/security.json
[junit4] 2> 6 /solr/configs/config
[junit4] 2> 3 /solr/collections/stateformat1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 18 /solr/collections/falsepredicate/state.json
[junit4] 2> 12 /solr/collections/waitforstate/state.json
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 7 /solr/collections/testcollection/state.json
[junit4] 2> 4 /solr/overseer_elect/election/98520546959360006-127.0.0.1:43452_solr-n_0000000000
[junit4] 2> 3 /solr/collections/stateformat1/state.json
[junit4] 2> 2 /solr/collections/currentstate/state.json
[junit4] 2> 2 /solr/collections/delayed/state.json
[junit4] 2> 2 /solr/collections/tobedeleted/state.json
[junit4] 2> 2 /solr/overseer_elect/election/98520546959360012-127.0.0.1:53851_solr-n_0000000003
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/collections
[junit4] 2> 5 /solr/live_nodes
[junit4] 2>
[junit4] 2> 240330 INFO (SUITE-TestCollectionStateWatchers-seed#[ED764CE729C1E289]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-6.6/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_ED764CE729C1E289-001
[junit4] 2> Aug 21, 2017 8:31:43 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene62): {}, docValues:{}, maxPointsInLeafNode=1631, maxMBSortInHeap=7.391862111056997, sim=RandomSimilarity(queryNorm=false,coord=yes): {}, locale=he-IL, timezone=Africa/Bangui
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=93937520,total=432013312
[junit4] 2> NOTE: All tests run in this JVM: [CeilingEvaluatorTest, TestToleratedUpdateError, JDBCStreamTest, GreaterThanEvaluatorTest, LargeVolumeBinaryJettyTest, NotEvaluatorTest, SolrPingTest, ShardParamsTest, SolrExampleStreamingTest, HyperbolicTangentEvaluatorTest, FieldEvaluatorTest, TermsResponseTest, ExternalHttpClientTest, TestUpdateRequestCodec, ConcatOperationTest, FieldAnalysisResponseTest, CloudSolrClientBuilderTest, TestEmbeddedSolrServer, TestZkConfigManager, DivideEvaluatorTest, TestDelegationTokenRequest, SolrExampleEmbeddedTest, LargeVolumeJettyTest, QueryResponseTest, TestCoreAdmin, TestLBHttpSolrClient, LBHttpSolrClientBuilderTest, TestCloudSolrClientConnections, GraphExpressionTest, SelectWithEvaluatorsTest, StreamExpressionToExpessionTest, StreamingTest, SolrZkClientTest, TestCollectionStateWatchers]
[junit4] Completed [129/130 (1!)] on J0 in 59.89s, 9 tests, 1 failure <<< FAILURES!
[...truncated 42647 lines...]