You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/06/26 09:06:23 UTC
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 248 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/248/
3 tests failed.
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Java heap space
Stack Trace:
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
at java.nio.charset.CharsetEncoder.encode(CharsetEncoder.java:795)
at java.nio.charset.Charset.encode(Charset.java:843)
at java.nio.charset.Charset.encode(Charset.java:863)
at org.apache.http.client.utils.URLEncodedUtils.urlEncode(URLEncodedUtils.java:532)
at org.apache.http.client.utils.URLEncodedUtils.encodeFormFields(URLEncodedUtils.java:652)
at org.apache.http.client.utils.URLEncodedUtils.format(URLEncodedUtils.java:404)
at org.apache.http.client.utils.URLEncodedUtils.format(URLEncodedUtils.java:382)
at org.apache.http.client.entity.UrlEncodedFormEntity.<init>(UrlEncodedFormEntity.java:75)
at org.apache.solr.client.solrj.impl.HttpSolrClient.fillContentStream(HttpSolrClient.java:513)
at org.apache.solr.client.solrj.impl.HttpSolrClient.createMethod(HttpSolrClient.java:420)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:974)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:990)
at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:228)
at org.apache.solr.cloud.CloudInspectUtil.compareResults(CloudInspectUtil.java:167)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:668)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv_stored_idx
Error Message:
Some docs had errors -- check logs expected:<0> but was:<6>
Stack Trace:
java.lang.AssertionError: Some docs had errors -- check logs expected:<0> but was:<6>
at __randomizedtesting.SeedInfo.seed([BC9584D1D2827C3F:B6B2D5F1ADF4DF68]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:342)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_dv_stored_idx(TestStressCloudBlindAtomicUpdates.java:221)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeLost
Error Message:
ComputePlanAction should have computed exactly 1 operation expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: ComputePlanAction should have computed exactly 1 operation expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([BC9584D1D2827C3F:3804A2F516819B9]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction.testNodeLost(TestComputePlanAction.java:173)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13564 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestComputePlanAction_BC9584D1D2827C3F-001/init-core-data-001
[junit4] 2> 949799 DEBUG (SUITE-TestComputePlanAction-seed#[BC9584D1D2827C3F]-worker) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10000_solr
[junit4] 2> 949800 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 949800 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 949801 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 949801 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 949806 INFO (SUITE-TestComputePlanAction-seed#[BC9584D1D2827C3F]-worker) [ ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 949808 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 949810 INFO (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 949811 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 949816 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 949816 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:177) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 949831 DEBUG (ScheduledTrigger-1329-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 949849 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 949849 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 949849 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 949849 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 949850 WARN (ScheduledTrigger-1329-thread-3) [ ] o.a.s.c.a.ScheduledTrigger ScheduledTrigger ran but was already closed
[junit4] 2> 949850 ERROR (ScheduledTrigger-1329-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Unexpected exception from trigger: .scheduled_maintenance
[junit4] 2> java.lang.RuntimeException: Trigger has been closed
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTrigger.run(ScheduledTrigger.java:155) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:589) [java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 949850 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 949850 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 949855 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 949855 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 949990 DEBUG (simCloudManagerPool-1333-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 950207 DEBUG (simCloudManagerPool-1333-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 950227 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion -1
[junit4] 2> 950227 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 950229 DEBUG (simCloudManagerPool-1333-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 950230 DEBUG (simCloudManagerPool-1333-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 950231 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 950247 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 950247 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 3
[junit4] 2> 950247 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 950247 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 950247 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 950247 DEBUG (ScheduledTrigger-1334-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 950268 DEBUG (ScheduledTrigger-1334-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 950288 DEBUG (ScheduledTrigger-1334-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 950309 DEBUG (ScheduledTrigger-1334-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 950329 DEBUG (ScheduledTrigger-1334-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 950349 DEBUG (ScheduledTrigger-1334-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950350 DEBUG (simCloudManagerPool-1333-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 950350 DEBUG (simCloudManagerPool-1333-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 950352 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 950352 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 950352 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 950352 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 950353 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950365 DEBUG (simCloudManagerPool-1333-thread-4) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeWithMultipleReplicasLost
[junit4] 2> 950370 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950373 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950381 DEBUG (simCloudManagerPool-1333-thread-4) [ ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 950384 DEBUG (simCloudManagerPool-1333-thread-4) [ ] o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 950390 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950393 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950410 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950413 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950430 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950433 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950450 INFO (simCloudManagerPool-1333-thread-4) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 30280746068 sessionWrapper.createTime 30280742758805522, this.sessionWrapper.createTime 30280742758805522
[junit4] 2> 950450 DEBUG (simCloudManagerPool-1333-thread-4) [ ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 950451 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950453 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950471 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950474 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950491 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950494 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950511 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950514 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950532 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950534 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950552 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950554 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950574 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950575 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950585 DEBUG (simCloudManagerPool-1333-thread-11) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeWithMultipleReplicasLost / shard2: {"core_node4":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testNodeWithMultipleReplicasLost"}}
[junit4] 2> 950594 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 950595 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 4
[junit4] 2> 950603 DEBUG (simCloudManagerPool-1333-thread-12) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeWithMultipleReplicasLost / shard1: {"core_node1":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeWithMultipleReplicasLost"}}
[junit4] 2> 950605 DEBUG (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction -- cluster state: znodeVersion: 5
[junit4] 2> live nodes:[127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> collections:{testNodeWithMultipleReplicasLost=DocCollection(testNodeWithMultipleReplicasLost//clusterstate.json/4)={
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{
[junit4] 2> "shard2":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node6":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard2_replica_n5",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active"},
[junit4] 2> "shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard1_replica_n3",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active"}}}}
[junit4] 2> 950614 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 950614 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10001_solr
[junit4] 2> 950615 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 950615 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10001_solr
[junit4] 2> 950635 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 950635 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 950635 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10001_solr]
[junit4] 2> 950635 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6b942f842d4b8aT7lb18b2cw2zb07m1ytmks54i0",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":30280754309974922,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[30280754309974922],
[junit4] 2> "nodeNames":["127.0.0.1:10001_solr"]}}
[junit4] 2> 950636 DEBUG (ScheduledTrigger-1334-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance, node_lost_trigger]
[junit4] 2> 950867 DEBUG (simCloudManagerPool-1333-thread-15) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
[junit4] 2> 950877 DEBUG (simCloudManagerPool-1333-thread-15) [ ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 950879 DEBUG (simCloudManagerPool-1333-thread-15) [ ] o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 950890 INFO (simCloudManagerPool-1333-thread-15) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 30280768055 sessionWrapper.createTime 30280767534649922, this.sessionWrapper.createTime 30280767534649922
[junit4] 2> 950890 DEBUG (simCloudManagerPool-1333-thread-15) [ ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 950920 DEBUG (simCloudManagerPool-1333-thread-15) [ ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
[junit4] 2> "replicas":{"core_node1":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}
[junit4] 2> 950927 DEBUG (AutoscalingActionExecutor-1335-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6b942f842d4b8aT7lb18b2cw2zb07m1ytmks54i0",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":30280754309974922,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[30280754309974922],
[junit4] 2> "_enqueue_time_":30280755352410822,
[junit4] 2> "nodeNames":["127.0.0.1:10001_solr"]}}
[junit4] 2> 950927 DEBUG (AutoscalingActionExecutor-1335-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"6b942f842d4b8aT7lb18b2cw2zb07m1ytmks54i0",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":30280754309974922,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[30280754309974922],
[junit4] 2> "_enqueue_time_":30280755352410822,
[junit4] 2> "nodeNames":["127.0.0.1:10001_solr"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
[junit4] 2> 950927 DEBUG (AutoscalingActionExecutor-1335-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 950959 DEBUG (AutoscalingActionExecutor-1335-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 950961 INFO (AutoscalingActionExecutor-1335-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10003_solr&inPlaceMove=true&replica=core_node5
[junit4] 2> 950971 INFO (AutoscalingActionExecutor-1335-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10000_solr&inPlaceMove=true&replica=core_node3
[junit4] 2> 950971 INFO (AutoscalingActionExecutor-1335-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 30280772118 sessionWrapper.createTime 30280771511191572, this.sessionWrapper.createTime 30280771511191572
[junit4] 2> 950971 DEBUG (AutoscalingActionExecutor-1335-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 950976 DEBUG (simCloudManagerPool-1333-thread-18) [ ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
[junit4] 2> "replicas":{"core_node1":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}
[junit4] 2> 950984 INFO (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10003_solr&inPlaceMove=true&replica=core_node5
[junit4] 2> 950984 INFO (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction action=MOVEREPLICA&collection=testNodeWithMultipleReplicasLost&targetNode=127.0.0.1:10000_solr&inPlaceMove=true&replica=core_node3
[junit4] 2> 950984 INFO (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction -------------_ FINAL STATE --------------
[junit4] 2> 950984 INFO (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction * Node values: {
[junit4] 2> "127.0.0.1:10001_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":2,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1022,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10001_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10001,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10000_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":1,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":999,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10000_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10000,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10002_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":2,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1022,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10002_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10002,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10003_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":2,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1022,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10003_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10003,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
[junit4] 2> 950985 INFO (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction * Live nodes: [127.0.0.1:10000_solr, 127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 950985 INFO (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction * Collection .system state: DocCollection(.system//clusterstate.json/9)={
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{"core_node1":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 950985 INFO (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction * Collection testNodeWithMultipleReplicasLost state: DocCollection(testNodeWithMultipleReplicasLost//clusterstate.json/9)={
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{
[junit4] 2> "shard2":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node6":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard2_replica_n6",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard2_replica_n4",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10000_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard2_replica_n5",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active"},
[junit4] 2> "shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard1_replica_n2",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeWithMultipleReplicasLost_shard1_replica_n3",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10001_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 950993 INFO (TEST-TestComputePlanAction.testNodeWithMultipleReplicasLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 3
[junit4] 2> ## Empty nodes: 0
[junit4] 2> ## Dead nodes: 1
[junit4] 2> ## 127.0.0.1:10001_solr
[junit4] 2> ## Collections: [.system, testNodeWithMultipleReplicasLost]
[junit4] 2> ## Max replicas per node: 2
[junit4] 2> ## Min replicas per node: 1
[junit4] 2> ## Total replicas: 5
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ## * testNodeWithMultipleReplicasLost 4
[junit4] 2> ## - active 4
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 2
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_lost_trigger
[junit4] 2> ## - BEFORE_ACTION 1
[junit4] 2> ## - STARTED 1
[junit4] 2>
[junit4] 2> 951089 DEBUG (simCloudManagerPool-1333-thread-19) [ ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
[junit4] 2> "replicas":{"core_node1":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}
[junit4] 2> 951157 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 951157 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10001_solr
[junit4] 2> 951158 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10002_solr
[junit4] 2> 951158 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.SimClusterStateProvider --- new Overseer leader: 127.0.0.1:10003_solr
[junit4] 2> 951158 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10000_solr
[junit4] 2> 951158 INFO (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 951158 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 951158 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 951159 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 951159 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 951159 WARN (AutoscalingActionExecutor-1335-thread-1) [ ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
[junit4] 2> java.io.IOException: java.lang.InterruptedException
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632) ~[test/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:790) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:757) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:322) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> Caused by: java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:630) ~[test/:?]
[junit4] 2> ... 10 more
[junit4] 2> 951160 DEBUG (AutoscalingActionExecutor-1335-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 11684 ms for event id=6b942f842d4b8aT7lb18b2cw2zb07m1ytmks54i0
[junit4] 2> 951160 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 951160 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 951160 DEBUG (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 951179 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 951179 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 951180 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 951180 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 951198 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10003_solr]
[junit4] 2> 951219 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 951219 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 951219 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951240 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951260 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951280 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951300 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951314 DEBUG (simCloudManagerPool-1336-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 951320 DEBUG (simCloudManagerPool-1336-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 951320 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 951321 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 951321 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 951321 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951338 DEBUG (simCloudManagerPool-1336-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 951338 DEBUG (simCloudManagerPool-1336-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 951341 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951345 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 951345 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 951345 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 951361 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951382 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951402 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951422 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951442 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951463 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951468 DEBUG (simCloudManagerPool-1336-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 951468 DEBUG (simCloudManagerPool-1336-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 951473 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 951474 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10003_solr]
[junit4] 2> 951474 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.sim.TestComputePlanAction$AssertingTriggerAction}], enabled=true}
[junit4] 2> 951474 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 951475 DEBUG (ScheduledTrigger-1337-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 951475 DEBUG (ScheduledTrigger-1337-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 951483 DEBUG (ScheduledTrigger-1337-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951493 DEBUG (simCloudManagerPool-1336-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 951493 DEBUG (simCloudManagerPool-1336-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 951494 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 951494 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 951496 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 951496 DEBUG (ScheduledTrigger-1337-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 951503 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951509 DEBUG (simCloudManagerPool-1336-thread-5) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeAdded
[junit4] 2> 951516 DEBUG (ScheduledTrigger-1337-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 951516 DEBUG (ScheduledTrigger-1337-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 951520 WARN (simCloudManagerPool-1336-thread-5) [ ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 4 on collection testNodeAdded is higher than the number of Solr instances currently live or live and part of your createNodeSet(1). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 951520 DEBUG (simCloudManagerPool-1336-thread-5) [ ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 951521 DEBUG (simCloudManagerPool-1336-thread-5) [ ] o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 951522 INFO (simCloudManagerPool-1336-thread-5) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 30280799676 sessionWrapper.createTime 30280799606090372, this.sessionWrapper.createTime 30280799606090372
[junit4] 2> 951522 DEBUG (simCloudManagerPool-1336-thread-5) [ ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 951526 DEBUG (ScheduledTrigger-1337-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951542 DEBUG (ScheduledTrigger-1337-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 951542 DEBUG (ScheduledTrigger-1337-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 951565 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951565 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 951566 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 951586 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951586 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 951586 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 951606 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 951606 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 951606 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 951606 DEBUG (simCloudManagerPool-1336-thread-10) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testNodeAdded / shard1: {"core_node2":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded"}}
[junit4] 2> 951615 DEBUG (simCloudManagerPool-1336-thread-11) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 951615 DEBUG (simCloudManagerPool-1336-thread-11) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 951619 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 951619 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 951626 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 951626 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 951626 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 951626 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10004_solr at time 30280804875708822
[junit4] 2> 951646 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 951647 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 951647 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 951647 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10004_solr] added at times [30280804875708822], now=30280805904298372
[junit4] 2> 951647 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6b943b4a212996T7lb18b2cw2zb07m1ytmks54i5",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":30280804875708822,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[30280804875708822],
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 951648 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, .scheduled_maintenance, node_added_trigger]
[junit4] 2> 951662 DEBUG (simCloudManagerPool-1336-thread-12) [ ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
[junit4] 2> 951803 DEBUG (simCloudManagerPool-1336-thread-12) [ ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 951803 DEBUG (simCloudManagerPool-1336-thread-12) [ ] o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 951804 INFO (simCloudManagerPool-1336-thread-12) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 30280813751 sessionWrapper.createTime 30280813719234372, this.sessionWrapper.createTime 30280813719234372
[junit4] 2> 951804 DEBUG (simCloudManagerPool-1336-thread-12) [ ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 951822 DEBUG (simCloudManagerPool-1336-thread-12) [ ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
[junit4] 2> "replicas":{"core_node2":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}
[junit4] 2> 951831 DEBUG (ScheduledTrigger-1337-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10004_solr
[junit4] 2> 951832 DEBUG (AutoscalingActionExecutor-1338-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6b943b4a212996T7lb18b2cw2zb07m1ytmks54i5",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":30280804875708822,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[30280804875708822],
[junit4] 2> "_enqueue_time_":30280805945024722,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 951832 DEBUG (AutoscalingActionExecutor-1338-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"6b943b4a212996T7lb18b2cw2zb07m1ytmks54i5",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":30280804875708822,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[30280804875708822],
[junit4] 2> "_enqueue_time_":30280805945024722,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
[junit4] 2> 951832 DEBUG (AutoscalingActionExecutor-1338-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper Creating a new session
[junit4] 2> 951833 DEBUG (AutoscalingActionExecutor-1338-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper New session created
[junit4] 2> 951833 INFO (AutoscalingActionExecutor-1338-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:10004_solr&inPlaceMove=true&replica=core_node2
[junit4] 2> 951835 INFO (AutoscalingActionExecutor-1338-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 30280815315 sessionWrapper.createTime 30280815190956422, this.sessionWrapper.createTime 30280815190956422
[junit4] 2> 951835 DEBUG (AutoscalingActionExecutor-1338-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper session set to NULL
[junit4] 2> 951847 DEBUG (simCloudManagerPool-1336-thread-15) [ ] o.a.s.c.a.s.SimClusterStateProvider -- no leader in shard1:{
[junit4] 2> "replicas":{"core_node2":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}
[junit4] 2> 951847 INFO (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction Node values: {
[junit4] 2> "127.0.0.1:10004_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":1,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1023,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10004_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10004,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10003_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":4,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":996,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10003_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10003,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
[junit4] 2> 951847 INFO (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction Live nodes: [127.0.0.1:10004_solr, 127.0.0.1:10003_solr], collection state: DocCollection(testNodeAdded//clusterstate.json/19)={
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"4",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n3",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n4",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 951848 INFO (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction -------------_ FINAL STATE --------------
[junit4] 2> 951848 INFO (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction * Node values: {
[junit4] 2> "127.0.0.1:10004_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":1,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":1023,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10004_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10004,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0},
[junit4] 2> "127.0.0.1:10003_solr":{
[junit4] 2> "sysprop.java.vendor":"Oracle Corporation",
[junit4] 2> "cores":4,
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "freedisk":996,
[junit4] 2> "sysprop.java.version":"1.8.0_172",
[junit4] 2> "ip_1":"127",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"1",
[junit4] 2> "node":"127.0.0.1:10003_solr",
[junit4] 2> "metrics:solr.jvm:buffers.direct.Count":0,
[junit4] 2> "port":10003,
[junit4] 2> "heapUsage":123450000,
[junit4] 2> "sysLoadAvg":1.0,
[junit4] 2> "metrics:solr.node:ADMIN./admin/authorization.clientErrors:count":0}}
[junit4] 2> 951848 INFO (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction * Live nodes: [127.0.0.1:10004_solr, 127.0.0.1:10003_solr]
[junit4] 2> 951848 INFO (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction * Collection .system state: DocCollection(.system//clusterstate.json/19)={
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{"core_node2":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 951848 INFO (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.TestComputePlanAction * Collection testNodeAdded state: DocCollection(testNodeAdded//clusterstate.json/19)={
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"4",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "autoCreated":"true",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "replicas":{
[junit4] 2> "core_node1":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node2":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "leader":"true",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n3",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n4",
[junit4] 2> "SEARCHER.searcher.maxDoc":0,
[junit4] 2> "SEARCHER.searcher.deletedDocs":0,
[junit4] 2> "INDEX.sizeInBytes":10240,
[junit4] 2> "node_name":"127.0.0.1:10003_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInGB":9.5367431640625E-6,
[junit4] 2> "SEARCHER.searcher.numDocs":0}},
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active"}}}
[junit4] 2> 951849 INFO (TEST-TestComputePlanAction.testNodeAdded-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 0
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: [.system, testNodeAdded]
[junit4] 2> ## Max replicas per node: 4
[junit4] 2> ## Min replicas per node: 1
[junit4] 2> ## Total replicas: 5
[junit4] 2> ## * .system 1
[junit4] 2> ## - active 1
[junit4] 2> ## * testNodeAdded 4
[junit4] 2> ## - active 4
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - CREATE 1
[junit4] 2> ## - autoscaling 5
[junit4] 2> ## - update 2
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger
[junit4] 2> ## - BEFORE_ACTION 1
[junit4] 2> ## - STARTED 1
[junit4] 2>
[junit4] 2> 951855 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 951855 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 951855 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 951856 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 951856 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
[junit4] 2> 951858 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.SimNodeStateProvider --removing value for 127.0.0.1:10004_solr
[junit4] 2> 951860 INFO (TEST-TestComputePlanAction.testNodeLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 951860 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:189) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> 951860 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 951860 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 951860 DEBUG (TEST-TestComputePlanAction.testNodeLost-seed#[BC9584D1D2827C3F]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 951867 WARN (AutoscalingActionExecutor-1338-thread-1) [ ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
[junit4] 2> java.io.IOException: java.lang.InterruptedException
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:632) ~[test/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:790) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:766) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:316) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_172]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_172]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_172]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_172]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[junit4] 2> Caused by: java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_172]
[junit4] 2> at java.util
[...truncated too long message...]
arent_f1_s=PostingsFormat(name=Memory), 30parent_f1_s=PostingsFormat(name=Memory), 14parent_f2_s=PostingsFormat(name=MockRandom), 20parent_f1_s=PostingsFormat(name=MockRandom), 1parent_f2_s=PostingsFormat(name=MockRandom), 7grand0_s=PostingsFormat(name=Memory), text_t=PostingsFormat(name=Asserting), 24parent_f2_s=PostingsFormat(name=Memory), 44parent_f2_s=Lucene50(blocksize=128), 34parent_f2_s=PostingsFormat(name=Asserting), 13parent_f2_s=PostingsFormat(name=Memory), type_s=PostingsFormat(name=Asserting), 23parent_f2_s=PostingsFormat(name=Asserting), 53parent_f2_s=PostingsFormat(name=Memory), 43parent_f2_s=PostingsFormat(name=MockRandom), 10parent_f1_s=Lucene50(blocksize=128), 33parent_f2_s=Lucene50(blocksize=128), 22parent_f2_s=Lucene50(blocksize=128), 4child0_s=PostingsFormat(name=MockRandom), 12parent_f2_s=PostingsFormat(name=Asserting), 52parent_f2_s=PostingsFormat(name=Asserting), 42parent_f2_s=PostingsFormat(name=Memory), 32parent_f2_s=PostingsFormat(name=MockRandom), 31parent_f1_s=PostingsFormat(name=MockRandom), 5grand1_s=PostingsFormat(name=Asserting), 6grand4_s=PostingsFormat(name=MockRandom), 11parent_f1_s=PostingsFormat(name=Asserting), 51parent_f1_s=PostingsFormat(name=Asserting), 54parent_f1_s=Lucene50(blocksize=128), 21parent_f1_s=Lucene50(blocksize=128), 4grand2_s=PostingsFormat(name=MockRandom), 7grand3_s=PostingsFormat(name=Asserting), 41parent_f1_s=PostingsFormat(name=Memory), 37parent_f1_s=PostingsFormat(name=Asserting), 40parent_f2_s=Lucene50(blocksize=128), 50parent_f2_s=PostingsFormat(name=MockRandom), 47parent_f1_s=Lucene50(blocksize=128), 14parent_f1_s=Lucene50(blocksize=128), 57parent_f1_s=PostingsFormat(name=MockRandom), 20parent_f2_s=PostingsFormat(name=Memory), 34parent_f1_s=PostingsFormat(name=Memory), 24parent_f1_s=PostingsFormat(name=MockRandom), 44parent_f1_s=PostingsFormat(name=Asserting), 30parent_f2_s=PostingsFormat(name=Asserting), 5parent_f1_s=PostingsFormat(name=Memory), 6grand0_s=PostingsFormat(name=MockRandom), 56parent_f2_s=PostingsFormat(name=Asserting), 46parent_f2_s=PostingsFormat(name=Memory), 10parent_f2_s=PostingsFormat(name=MockRandom), 8parent_f1_s=PostingsFormat(name=MockRandom), 27parent_f2_s=PostingsFormat(name=Asserting), 2parent_f2_s=PostingsFormat(name=Memory), 4child1_s=Lucene50(blocksize=128), 2grand1_s=PostingsFormat(name=Memory), 17parent_f2_s=PostingsFormat(name=Memory), 3child2_s=Lucene50(blocksize=128), 16parent_f2_s=PostingsFormat(name=MockRandom), 26parent_f2_s=Lucene50(blocksize=128), 1child0_s=Lucene50(blocksize=128), 36parent_f2_s=PostingsFormat(name=MockRandom), 7grand6_s=Lucene50(blocksize=128), 18parent_f1_s=Lucene50(blocksize=128), 28parent_f1_s=PostingsFormat(name=Asserting), 5grand0_s=Lucene50(blocksize=128), 6grand5_s=PostingsFormat(name=Memory), 7grand2_s=Lucene50(blocksize=128), 48parent_f1_s=PostingsFormat(name=Asserting), 15parent_f1_s=PostingsFormat(name=Asserting), 4grand3_s=PostingsFormat(name=Memory), 58parent_f1_s=Lucene50(blocksize=128), 6parent_f1_s=PostingsFormat(name=Asserting), 25parent_f1_s=Lucene50(blocksize=128), 0parent_f1_s=PostingsFormat(name=MockRandom), 3grand2_s=Lucene50(blocksize=128), 35parent_f1_s=PostingsFormat(name=MockRandom), 19parent_f2_s=Lucene50(blocksize=128), 4parent_f2_s=Lucene50(blocksize=128), 37parent_f2_s=Lucene50(blocksize=128), 47parent_f2_s=PostingsFormat(name=MockRandom), 57parent_f2_s=PostingsFormat(name=Memory), 38parent_f1_s=PostingsFormat(name=Memory), 5grand4_s=Lucene50(blocksize=128), 6grand1_s=PostingsFormat(name=Memory), 5parent_f2_s=PostingsFormat(name=MockRandom), 46parent_f1_s=PostingsFormat(name=MockRandom), 16parent_f1_s=PostingsFormat(name=Memory), 56parent_f1_s=PostingsFormat(name=Memory), 8parent_f2_s=Lucene50(blocksize=128), 39parent_f2_s=PostingsFormat(name=Memory), 29parent_f2_s=PostingsFormat(name=Asserting), 2parent_f1_s=PostingsFormat(name=Asserting), 4child2_s=PostingsFormat(name=Asserting), rnd_b=PostingsFormat(name=Memory), 27parent_f1_s=PostingsFormat(name=Memory), 45parent_f1_s=PostingsFormat(name=Memory), 17parent_f1_s=PostingsFormat(name=Asserting), 2grand0_s=PostingsFormat(name=Asserting), 49parent_f2_s=PostingsFormat(name=Asserting), 3child1_s=PostingsFormat(name=MockRandom), 55parent_f1_s=PostingsFormat(name=Asserting), 59parent_f2_s=Lucene50(blocksize=128), 7parent_f2_s=PostingsFormat(name=Asserting), 26parent_f1_s=PostingsFormat(name=Asserting), 2child0_s=PostingsFormat(name=MockRandom), 3parent_f1_s=Lucene50(blocksize=128), 36parent_f1_s=Lucene50(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Memory), id=DocValuesFormat(name=Asserting), text_t=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=181, maxMBSortInHeap=5.142025435293219, sim=RandomSimilarity(queryNorm=true): {}, locale=zh-CN, timezone=America/Catamarca
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=120884656,total=469237760
[junit4] 2> NOTE: All tests run in this JVM: [TestSolrXml, ExitableDirectoryReaderTest, RankQueryTest, TestSolrCoreParser, DistributedFacetPivotLargeTest, RuleEngineTest, TestUniqueKeyFieldResource, SystemLogListenerTest, TestSystemCollAutoCreate, LoggingHandlerTest, TestStressUserVersions, BadComponentTest, CurrencyFieldTypeTest, TestDocTermOrds, SignatureUpdateProcessorFactoryTest, OverseerTest, SolrRrdBackendFactoryTest, TestCSVLoader, TestTrieFacet, TestIntervalFaceting, MetricsHandlerTest, DistributedDebugComponentTest, FullHLLTest, GraphQueryTest, TestDistribDocBasedVersion, TestComplexPhraseLeadingWildcard, TestJsonFacets, TestGraphTermsQParserPlugin, TestTolerantUpdateProcessorCloud, TestTolerantSearch, TestCopyFieldCollectionResource, OverriddenZkACLAndCredentialsProvidersTest, TestLeaderElectionZkExpiry, TestManagedSchemaAPI, TestUnInvertedFieldException, DefaultValueUpdateProcessorTest, ZkControllerTest, TestDynamicLoading, HdfsRecoveryZkTest, TestScoreJoinQPNoScore, TestComputePlanAction, SoftAutoCommitTest, TestMultiValuedNumericRangeQuery, TestSQLHandlerNonCloud, TestInfoStreamLogging, TestClusterProperties, PKIAuthenticationIntegrationTest, EmbeddedSolrNoSerializeTest, OverseerRolesTest, UpdateParamsTest, TestSolrQueryResponse, TestShardHandlerFactory, DocumentAnalysisRequestHandlerTest, BasicDistributedZkTest, HdfsWriteToMultipleCollectionsTest, DeleteNodeTest, TestStressInPlaceUpdates, IndexSchemaTest, TestFilteredDocIdSet, TestReqParamsAPI, TestFieldCollectionResource, TestConfigOverlay, DistanceUnitsTest, TestRandomRequestDistribution, DocValuesNotIndexedTest, TestRandomCollapseQParserPlugin, BufferStoreTest, TestCollectionAPI, TestSSLRandomization, TestFunctionQuery, TestNumericTerms32, TestLegacyFieldReuse, RestoreTriggerStateTest, TestReversedWildcardFilterFactory, ActionThrottleTest, MaxSizeAutoCommitTest, MoreLikeThisHandlerTest, DistributedQueryComponentCustomSortTest, TestInitQParser, TaggerTest, TestSolrQueryParser, TestNodeLostTrigger, BlobRepositoryCloudTest, TestSizeLimitedDistributedMap, ClassificationUpdateProcessorIntegrationTest, TestSortByMinMaxFunction, DisMaxRequestHandlerTest, TestSolrCloudWithKerberosAlt, EnumFieldTest, MinimalSchemaTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestBinaryResponseWriter, CreateRoutedAliasTest, TestCustomSort, TestLRUStatsCache, HighlighterTest, DistributedFacetPivotSmallTest, TestSolrCLIRunExample, CircularListTest, TestDistributedSearch, TestAnalyzedSuggestions, TestPivotHelperCode, SparseHLLTest, TestRawResponseWriter, TestNonDefinedSimilarityFactory, OrderedExecutorTest, TestConfigSets, CdcrRequestHandlerTest, SchemaVersionSpecificBehaviorTest, TestPullReplica, BlockJoinFacetDistribTest, DistribJoinFromCollectionTest, MergeStrategyTest, NotRequiredUniqueKeyTest, TestDistribIDF, CloudMLTQParserTest, FacetPivotSmallTest, TestObjectReleaseTracker, TestSweetSpotSimilarityFactory, CdcrBootstrapTest, TestLegacyNumericRangeQueryBuilder, TestSearchPerf, SolrSlf4jReporterTest, PrimitiveFieldTypeTest, TestSortableTextField, CollectionStateFormat2Test, DocValuesMissingTest, NodeLostTriggerIntegrationTest, SecurityConfHandlerTest, SolrIndexMetricsTest, RequestHandlersTest, LeaderVoteWaitTimeoutTest, AnalysisAfterCoreReloadTest, BasicFunctionalityTest, ConvertedLegacyTest, SampleTest, SolrInfoBeanTest, SolrTestCaseJ4Test, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedGrouping, TestGroupingSearch, TestRandomDVFaceting, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, AliasIntegrationTest, CleanupOldIndexTest, CloudExitableDirectoryReaderTest, ConcurrentCreateRoutedAliasTest, ConfigSetsAPITest, ConnectionManagerTest, DeleteStatusTest, FullSolrCloudDistribCmdsTest]
[junit4] Completed [537/819 (3!)] on J1 in 2668.99s, 1 test, 1 error <<< FAILURES!
[...truncated 1014 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/temp/junit4-J1-20180626_065017_7827010339845398400990.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] java.lang.OutOfMemoryError: Java heap space
[junit4] Dumping heap to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/heapdumps/java_pid29211.hprof ...
[junit4] Heap dump file created [460370217 bytes in 14.582 secs]
[junit4] <<< JVM J1: EOF ----
[...truncated 9439 lines...]
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:651: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/build.xml:585: Some of the tests produced a heap dump, but did not fail. Maybe a suppressed OutOfMemoryError? Dumps created:
* java_pid29211.hprof
Total time: 394 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 251 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/251/
4 tests failed.
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
KeeperErrorCode = Session expired for /clusterstate.json
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /clusterstate.json
at __randomizedtesting.SeedInfo.seed([EEE7D450EC6464E0:66B3EB8A42980918]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:341)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:341)
at org.apache.solr.common.cloud.ZkStateReader.refreshLegacyClusterState(ZkStateReader.java:567)
at org.apache.solr.common.cloud.ZkStateReader.forceUpdateCollection(ZkStateReader.java:371)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:681)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:676)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:471)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:341)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1006)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.core.TestLazyCores.testLazyLoad
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([EEE7D450EC6464E0]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.core.TestLazyCores
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([EEE7D450EC6464E0]:0)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_stored_idx
Error Message:
Some docs had errors -- check logs expected:<0> but was:<13>
Stack Trace:
java.lang.AssertionError: Some docs had errors -- check logs expected:<0> but was:<13>
at __randomizedtesting.SeedInfo.seed([EEE7D450EC6464E0:FE8679A9F9C621CE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:342)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_stored_idx(TestStressCloudBlindAtomicUpdates.java:241)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13183 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> 113157 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[EEE7D450EC6464E0]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/init-core-data-001
[junit4] 2> 113158 WARN (SUITE-FullSolrCloudDistribCmdsTest-seed#[EEE7D450EC6464E0]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2518 numCloses=2518
[junit4] 2> 113158 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[EEE7D450EC6464E0]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 113160 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[EEE7D450EC6464E0]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 113160 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[EEE7D450EC6464E0]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /jxy/xz
[junit4] 2> 113242 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 113304 INFO (Thread-2) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 113304 INFO (Thread-2) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 113712 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.ZkTestServer start zk server on port:33838
[junit4] 2> 113994 ERROR (Thread-2) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 114252 INFO (zkConnectionManagerCallback-4-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 114462 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x102f32584630000, likely client has closed socket
[junit4] 2> 114499 INFO (zkConnectionManagerCallback-6-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 114616 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 114667 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 114681 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 114750 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 114762 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 114764 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 114766 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 114768 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 114784 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 114821 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 114836 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 114855 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 118522 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 118914 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 118914 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 118918 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 119025 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@a79f47a{/jxy/xz,null,AVAILABLE}
[junit4] 2> 119246 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@11df4930{HTTP/1.1,[http/1.1]}{127.0.0.1:39480}
[junit4] 2> 119246 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.Server Started @119489ms
[junit4] 2> 119247 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/tempDir-001/control/data, replicaType=NRT, hostContext=/jxy/xz, hostPort=39480, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/control-001/cores}
[junit4] 2> 119500 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.u.c.SSLCredentialProviderFactory Processing SSL Credential Provider chain: env;sysprop
[junit4] 2> 119536 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 119537 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 119552 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.5.0
[junit4] 2> 119572 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 119572 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 119573 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-30T04:11:01.707Z
[junit4] 2> 119574 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 119848 INFO (zkConnectionManagerCallback-8-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 119869 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 119872 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/control-001/solr.xml
[junit4] 2> 119956 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 119956 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 120117 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 123089 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33838/solr
[junit4] 2> 123305 INFO (zkConnectionManagerCallback-12-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 123411 INFO (zkConnectionManagerCallback-14-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 124973 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 125007 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39480_jxy%2Fxz
[junit4] 2> 125009 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.c.Overseer Overseer (id=72887886451769348-127.0.0.1:39480_jxy%2Fxz-n_0000000000) starting
[junit4] 2> 125519 INFO (zkConnectionManagerCallback-21-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 125572 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33838/solr ready
[junit4] 2> 125973 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39480_jxy%2Fxz
[junit4] 2> 126257 INFO (zkCallback-20-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 126323 INFO (zkCallback-13-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 126654 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 126838 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 126839 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 126839 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 126856 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/control-001/cores
[junit4] 2> 127939 INFO (zkConnectionManagerCallback-27-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 128102 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 128145 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33838/solr ready
[junit4] 2> 129882 INFO (qtp1125608235-44) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:39480_jxy%252Fxz&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 130027 INFO (OverseerThreadFactory-26-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 130304 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz x:control_collection_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 130316 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz x:control_collection_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 131481 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 131895 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 131983 WARN (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.CurrencyField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 132949 WARN (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.GeoHashField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 132986 WARN (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly.
[junit4] 2> 133149 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 133597 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 133599 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 133599 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 133599 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 133652 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 133752 WARN (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 134566 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 134566 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 134930 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 134930 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 134947 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@3de3466
[junit4] 2> 134963 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@64907b52[control_collection_shard1_replica_n1] main]
[junit4] 2> 135035 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 135056 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 135057 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 135094 INFO (searcherExecutor-31-thread-1-processing-n:127.0.0.1:39480_jxy%2Fxz x:control_collection_shard1_replica_n1 c:control_collection s:shard1) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@64907b52[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 135094 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604669278496227328
[junit4] 2> 135426 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/control_collection/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 135663 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 135663 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 135663 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39480/jxy/xz/control_collection_shard1_replica_n1/
[junit4] 2> 135679 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 135698 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39480/jxy/xz/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 135698 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 136047 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39480/jxy/xz/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 136139 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 136177 INFO (qtp1125608235-38) [n:127.0.0.1:39480_jxy%2Fxz c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=5895
[junit4] 2> 136301 INFO (zkCallback-13-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 136627 INFO (qtp1125608235-44) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 136627 INFO (qtp1125608235-44) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:39480_jxy%252Fxz&wt=javabin&version=2} status=0 QTime=6761
[junit4] 2> 136861 INFO (zkConnectionManagerCallback-32-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 136988 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 137024 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33838/solr ready
[junit4] 2> 137027 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 137086 INFO (qtp1125608235-40) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 137252 INFO (OverseerThreadFactory-26-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 137309 WARN (OverseerThreadFactory-26-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 137325 INFO (OverseerCollectionConfigSetProcessor-72887886451769348-127.0.0.1:39480_jxy%2Fxz-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 138034 INFO (qtp1125608235-40) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 138034 INFO (qtp1125608235-40) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=949
[junit4] 2> 138107 INFO (OverseerCollectionConfigSetProcessor-72887886451769348-127.0.0.1:39480_jxy%2Fxz-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 139146 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-1-001 of type NRT
[junit4] 2> 139147 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 139247 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 139247 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 139247 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 139264 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3b25db87{/jxy/xz,null,AVAILABLE}
[junit4] 2> 139264 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1e522570{HTTP/1.1,[http/1.1]}{127.0.0.1:41537}
[junit4] 2> 139264 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.Server Started @139507ms
[junit4] 2> 139264 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/jxy/xz, hostPort=41537, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-1-001/cores}
[junit4] 2> 139265 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 139265 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 139265 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.5.0
[junit4] 2> 139265 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 139265 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 139265 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-30T04:11:21.399Z
[junit4] 2> 139312 INFO (zkConnectionManagerCallback-34-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 139401 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 139401 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-1-001/solr.xml
[junit4] 2> 139437 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 139437 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 139452 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 140677 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33838/solr
[junit4] 2> 140763 INFO (zkConnectionManagerCallback-38-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 141035 INFO (zkConnectionManagerCallback-40-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 141165 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41537_jxy%2Fxz ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 141200 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41537_jxy%2Fxz ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 141255 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41537_jxy%2Fxz ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 141255 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41537_jxy%2Fxz ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41537_jxy%2Fxz
[junit4] 2> 141356 INFO (zkCallback-20-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 141377 INFO (zkCallback-31-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 141377 INFO (zkCallback-13-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 141739 INFO (zkCallback-39-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 141775 INFO (zkConnectionManagerCallback-47-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 141934 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41537_jxy%2Fxz ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 141936 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41537_jxy%2Fxz ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33838/solr ready
[junit4] 2> 141966 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41537_jxy%2Fxz ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 142261 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41537_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 142316 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41537_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 142348 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41537_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 142351 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41537_jxy%2Fxz ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-1-001/cores
[junit4] 2> 142658 INFO (qtp1125608235-39) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:41537_jxy%252Fxz&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 142866 INFO (OverseerThreadFactory-26-thread-3) [ c:collection1 s:shard2 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:41537_jxy%2Fxz for creating new replica
[junit4] 2> 143319 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz x:collection1_shard2_replica_n61] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 144453 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 144597 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema [collection1_shard2_replica_n61] Schema name=test
[junit4] 2> 145243 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 145562 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n61' using configuration from collection collection1, trusted=true
[junit4] 2> 145563 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n61' (registry 'solr.core.collection1.shard2.replica_n61') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 145563 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 145563 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [[collection1_shard2_replica_n61] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-1-001/cores/collection1_shard2_replica_n61], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-1-001/cores/collection1_shard2_replica_n61/data/]
[junit4] 2> 145618 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 145635 WARN (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 146205 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 146205 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 146207 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 146207 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 146225 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@73e63a93
[junit4] 2> 146226 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.SolrIndexSearcher Opening [Searcher@7eea4a5e[collection1_shard2_replica_n61] main]
[junit4] 2> 146234 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 146235 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 146235 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 146236 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604669290179461120
[junit4] 2> 146401 INFO (searcherExecutor-45-thread-1-processing-n:127.0.0.1:41537_jxy%2Fxz x:collection1_shard2_replica_n61 c:collection1 s:shard2) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [collection1_shard2_replica_n61] Registered new searcher Searcher@7eea4a5e[collection1_shard2_replica_n61] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 146419 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard2 to Terms{values={core_node62=0}, version=0}
[junit4] 2> 146536 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 146536 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 146536 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41537/jxy/xz/collection1_shard2_replica_n61/
[junit4] 2> 146536 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 146536 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy http://127.0.0.1:41537/jxy/xz/collection1_shard2_replica_n61/ has no replicas
[junit4] 2> 146536 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 146553 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41537/jxy/xz/collection1_shard2_replica_n61/ shard2
[junit4] 2> 146663 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 146667 INFO (qtp1030920798-99) [n:127.0.0.1:41537_jxy%2Fxz c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3348
[junit4] 2> 146799 INFO (qtp1125608235-39) [n:127.0.0.1:39480_jxy%2Fxz c:collection1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:41537_jxy%252Fxz&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=4170
[junit4] 2> 147001 INFO (OverseerCollectionConfigSetProcessor-72887886451769348-127.0.0.1:39480_jxy%2Fxz-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 148102 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-2-001 of type NRT
[junit4] 2> 148122 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 148141 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 148141 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 148141 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 148142 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@64020b00{/jxy/xz,null,AVAILABLE}
[junit4] 2> 148142 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3f7eeb88{HTTP/1.1,[http/1.1]}{127.0.0.1:41726}
[junit4] 2> 148142 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.Server Started @148385ms
[junit4] 2> 148142 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/jxy/xz, hostPort=41726, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-2-001/cores}
[junit4] 2> 148143 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 148161 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 148161 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.5.0
[junit4] 2> 148161 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 148161 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 148161 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-30T04:11:30.295Z
[junit4] 2> 148413 INFO (zkConnectionManagerCallback-50-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 148419 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 148419 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-2-001/solr.xml
[junit4] 2> 148492 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 148492 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 148507 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 149202 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33838/solr
[junit4] 2> 149299 INFO (zkConnectionManagerCallback-54-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 149349 INFO (zkConnectionManagerCallback-56-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 149401 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41726_jxy%2Fxz ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 149437 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41726_jxy%2Fxz ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 149440 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41726_jxy%2Fxz ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 149440 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41726_jxy%2Fxz ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41726_jxy%2Fxz
[junit4] 2> 149497 INFO (zkCallback-13-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 149506 INFO (zkCallback-39-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 149534 INFO (zkCallback-31-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 149555 INFO (zkCallback-46-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 149556 INFO (zkCallback-55-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 149531 INFO (zkCallback-20-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 149912 INFO (zkConnectionManagerCallback-63-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 149950 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41726_jxy%2Fxz ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 149952 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41726_jxy%2Fxz ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33838/solr ready
[junit4] 2> 150022 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41726_jxy%2Fxz ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 150253 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41726_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 150379 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41726_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 150379 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41726_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 150413 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:41726_jxy%2Fxz ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-2-001/cores
[junit4] 2> 150793 INFO (qtp1125608235-42) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:41726_jxy%252Fxz&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 150908 INFO (OverseerThreadFactory-26-thread-4) [ c:collection1 s:shard3 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:41726_jxy%2Fxz for creating new replica
[junit4] 2> 151048 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz x:collection1_shard3_replica_n63] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n63&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 158194 WARN (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]-SendThread(127.0.0.1:33838)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 7013ms for sessionid 0x102f3258463000f
[junit4] 2> 158194 WARN (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]-SendThread(127.0.0.1:33838)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 7042ms for sessionid 0x102f3258463000b
[junit4] 2> 158259 WARN (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]-SendThread(127.0.0.1:33838)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 7107ms for sessionid 0x102f32584630005
[junit4] 2> 158267 WARN (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]-SendThread(127.0.0.1:33838)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 7136ms for sessionid 0x102f32584630007
[junit4] 2> 158267 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x102f3258463000b, likely client has closed socket
[junit4] 2> 158268 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x102f32584630005, likely client has closed socket
[junit4] 2> 158268 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x102f32584630007, likely client has closed socket
[junit4] 2> 158268 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x102f3258463000f, likely client has closed socket
[junit4] 2> 158276 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 158348 WARN (zkConnectionManagerCallback-63-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7b27e64d name: ZooKeeperConnection Watcher:127.0.0.1:33838/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 158348 WARN (zkConnectionManagerCallback-63-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 158373 WARN (zkConnectionManagerCallback-21-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@56ce51af name: ZooKeeperConnection Watcher:127.0.0.1:33838/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 158373 WARN (zkConnectionManagerCallback-21-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 158425 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema [collection1_shard3_replica_n63] Schema name=test
[junit4] 2> 158432 WARN (zkConnectionManagerCallback-47-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@2eecaeac name: ZooKeeperConnection Watcher:127.0.0.1:33838/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 158432 WARN (zkConnectionManagerCallback-47-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 158466 WARN (zkConnectionManagerCallback-32-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@19b9cf70 name: ZooKeeperConnection Watcher:127.0.0.1:33838/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 158466 WARN (zkConnectionManagerCallback-32-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 159251 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 159470 INFO (zkConnectionManagerCallback-32-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 159676 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard3_replica_n63' using configuration from collection collection1, trusted=true
[junit4] 2> 159709 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard3.replica_n63' (registry 'solr.core.collection1.shard3.replica_n63') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 159710 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 159710 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore [[collection1_shard3_replica_n63] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-2-001/cores/collection1_shard3_replica_n63], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-2-001/cores/collection1_shard3_replica_n63/data/]
[junit4] 2> 159731 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=37, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 159784 WARN (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 160176 INFO (zkConnectionManagerCallback-21-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 160194 INFO (zkConnectionManagerCallback-63-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 160415 INFO (zkConnectionManagerCallback-47-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 160521 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 160521 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 160523 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 160523 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 160525 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@5f31de38
[junit4] 2> 160558 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.SolrIndexSearcher Opening [Searcher@751581b0[collection1_shard3_replica_n63] main]
[junit4] 2> 160571 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 160571 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 160572 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 160572 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604669305211846656
[junit4] 2> 160685 INFO (searcherExecutor-59-thread-1-processing-n:127.0.0.1:41726_jxy%2Fxz x:collection1_shard3_replica_n63 c:collection1 s:shard3) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore [collection1_shard3_replica_n63] Registered new searcher Searcher@751581b0[collection1_shard3_replica_n63] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 160787 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ZkShardTerms Successful update of terms at /collections/collection1/terms/shard3 to Terms{values={core_node64=0}, version=0}
[junit4] 2> 160866 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 160866 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 160866 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41726/jxy/xz/collection1_shard3_replica_n63/
[junit4] 2> 160867 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 160867 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy http://127.0.0.1:41726/jxy/xz/collection1_shard3_replica_n63/ has no replicas
[junit4] 2> 160867 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 161026 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41726/jxy/xz/collection1_shard3_replica_n63/ shard3
[junit4] 2> 161078 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 161109 INFO (qtp1391152452-139) [n:127.0.0.1:41726_jxy%2Fxz c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n63&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=10061
[junit4] 2> 161262 INFO (qtp1125608235-42) [n:127.0.0.1:39480_jxy%2Fxz c:collection1 ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:41726_jxy%252Fxz&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2} status=0 QTime=10469
[junit4] 2> 162382 INFO (OverseerCollectionConfigSetProcessor-72887886451769348-127.0.0.1:39480_jxy%2Fxz-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 162484 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-3-001 of type NRT
[junit4] 2> 162486 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 162529 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 162529 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 162529 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 162529 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5ebb633e{/jxy/xz,null,AVAILABLE}
[junit4] 2> 162530 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@45920107{HTTP/1.1,[http/1.1]}{127.0.0.1:40833}
[junit4] 2> 162530 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.e.j.s.Server Started @162773ms
[junit4] 2> 162530 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/jxy/xz, hostPort=40833, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-3-001/cores}
[junit4] 2> 162530 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 162530 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 162531 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.5.0
[junit4] 2> 162531 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 162531 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 162531 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-30T04:11:44.665Z
[junit4] 2> 162621 INFO (zkConnectionManagerCallback-66-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 162624 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 162624 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-3-001/solr.xml
[junit4] 2> 162640 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 162641 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 162675 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 163601 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33838/solr
[junit4] 2> 163772 INFO (zkConnectionManagerCallback-70-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 164002 INFO (zkConnectionManagerCallback-72-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 164043 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:40833_jxy%2Fxz ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 164081 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:40833_jxy%2Fxz ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 164101 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:40833_jxy%2Fxz ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 164101 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:40833_jxy%2Fxz ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40833_jxy%2Fxz
[junit4] 2> 164165 INFO (zkCallback-62-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 164327 INFO (zkCallback-71-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 164384 INFO (zkCallback-55-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 164396 INFO (zkCallback-39-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 164396 INFO (zkCallback-46-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 164400 INFO (zkCallback-20-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 164400 INFO (zkCallback-31-thread-4) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 164400 INFO (zkCallback-13-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 164700 INFO (zkConnectionManagerCallback-79-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 164828 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:40833_jxy%2Fxz ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 164829 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:40833_jxy%2Fxz ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33838/solr ready
[junit4] 2> 164842 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:40833_jxy%2Fxz ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 164970 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:40833_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 165008 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:40833_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 165008 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:40833_jxy%2Fxz ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 165023 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[EEE7D450EC6464E0]) [n:127.0.0.1:40833_jxy%2Fxz ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-3-001/cores
[junit4] 2> 165387 INFO (qtp1125608235-39) [n:127.0.0.1:39480_jxy%2Fxz ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:40833_jxy%252Fxz&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 165478 INFO (OverseerThreadFactory-26-thread-5) [ c:collection1 s:shard1 ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:40833_jxy%2Fxz for creating new replica
[junit4] 2> 165526 INFO (qtp1651179673-187) [n:127.0.0.1:40833_jxy%2Fxz x:collection1_shard1_replica_n65] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n65&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 166803 INFO (qtp1651179673-187) [n:127.0.0.1:40833_jxy%2Fxz c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 167078 INFO (qtp1651179673-187) [n:127.0.0.1:40833_jxy%2Fxz c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.IndexSchema [collection1_shard1_replica_n65] Schema name=test
[junit4] 2> 168016 INFO (qtp1651179673-187) [n:127.0.0.1:40833_jxy%2Fxz c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 168216 INFO (qtp1651179673-187) [n:127.0.0.1:40833_jxy%2Fxz c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n65' using configuration from collection collection1, trusted=true
[junit4] 2> 168269 INFO (qtp1651179673-187) [n:127.0.0.1:40833_jxy%2Fxz c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n65' (registry 'solr.core.collection1.shard1.replica_n65') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7a1d634e
[junit4] 2> 168270 INFO (qtp1651179673-187) [n:127.0.0.1:40833_jxy%2Fxz c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 168270 INFO (qtp1651179673-187) [n:127.0.0.1:40833_jxy%2Fxz c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrCore [[collection1_shard1_replica_n65] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_EEE7D450EC6464E0-001/shard-3-001/cor
[...truncated too long message...]
lity-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null381244814
[copy] Copying 39 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null381244814
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null381244814
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1143456647
[copy] Copying 247 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1143456647
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1143456647
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 28ms :: artifacts dl 5ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 589 minutes 18 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 250 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/250/
6 tests failed.
FAILED: org.apache.solr.TestCrossCoreJoin.testJoin
Error Message:
mismatch: '1'!='5' @ response/docs/[0]/id
Stack Trace:
java.lang.RuntimeException: mismatch: '1'!='5' @ response/docs/[0]/id
at __randomizedtesting.SeedInfo.seed([B342DA41149789B6:8E71EFCB75C08E20]:0)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1005)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:952)
at org.apache.solr.TestCrossCoreJoin.doTestJoin(TestCrossCoreJoin.java:88)
at org.apache.solr.TestCrossCoreJoin.testJoin(TestCrossCoreJoin.java:79)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.TestCrossCoreJoin.testScoreJoin
Error Message:
mismatch: '1'!='5' @ response/docs/[0]/id
Stack Trace:
java.lang.RuntimeException: mismatch: '1'!='5' @ response/docs/[0]/id
at __randomizedtesting.SeedInfo.seed([B342DA41149789B6:B6ACDD5F43667734]:0)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:1005)
at org.apache.solr.SolrTestCaseJ4.assertJQ(SolrTestCaseJ4.java:952)
at org.apache.solr.TestCrossCoreJoin.doTestJoin(TestCrossCoreJoin.java:88)
at org.apache.solr.TestCrossCoreJoin.testScoreJoin(TestCrossCoreJoin.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.LIROnShardRestartTest.testAllReplicasInLIR
Error Message:
No live SolrServers available to handle this request
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
at __randomizedtesting.SeedInfo.seed([B342DA41149789B6:E9DAE0876A17EE51]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:460)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.LIROnShardRestartTest.testAllReplicasInLIR(LIROnShardRestartTest.java:175)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.LIROnShardRestartTest.testSeveralReplicasInLIR
Error Message:
No live SolrServers available to handle this request
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request
at __randomizedtesting.SeedInfo.seed([B342DA41149789B6:35184D7534CDC4EF]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:460)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.LIROnShardRestartTest.testSeveralReplicasInLIR(LIROnShardRestartTest.java:190)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_stored_idx
Error Message:
Some docs had errors -- check logs expected:<0> but was:<20>
Stack Trace:
java.lang.AssertionError: Some docs had errors -- check logs expected:<0> but was:<20>
at __randomizedtesting.SeedInfo.seed([B342DA41149789B6:A32377B80135CC98]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.checkField(TestStressCloudBlindAtomicUpdates.java:342)
at org.apache.solr.cloud.TestStressCloudBlindAtomicUpdates.test_stored_idx(TestStressCloudBlindAtomicUpdates.java:241)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.api.collections.TestLocalFSCloudBackupRestore.test
Error Message:
expected:<COMPLETED> but was:<FAILED>
Stack Trace:
java.lang.AssertionError: expected:<COMPLETED> but was:<FAILED>
at __randomizedtesting.SeedInfo.seed([B342DA41149789B6:3B16E59BBA6BE44E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:327)
at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13021 lines...]
[junit4] Suite: org.apache.solr.cloud.LIROnShardRestartTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/init-core-data-001
[junit4] 2> 175781 WARN (SUITE-LIROnShardRestartTest-seed#[B342DA41149789B6]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 175784 INFO (SUITE-LIROnShardRestartTest-seed#[B342DA41149789B6]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 175786 INFO (SUITE-LIROnShardRestartTest-seed#[B342DA41149789B6]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 175786 INFO (SUITE-LIROnShardRestartTest-seed#[B342DA41149789B6]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 175828 INFO (SUITE-LIROnShardRestartTest-seed#[B342DA41149789B6]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001
[junit4] 2> 175828 INFO (SUITE-LIROnShardRestartTest-seed#[B342DA41149789B6]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 175857 INFO (Thread-174) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 175857 INFO (Thread-174) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 175889 ERROR (Thread-174) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 175957 INFO (SUITE-LIROnShardRestartTest-seed#[B342DA41149789B6]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:44897
[junit4] 2> 176062 INFO (zkConnectionManagerCallback-255-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 176068 INFO (jetty-launcher-252-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 176069 INFO (jetty-launcher-252-thread-2) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 176069 INFO (jetty-launcher-252-thread-3) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 176126 INFO (jetty-launcher-252-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 176126 INFO (jetty-launcher-252-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 176126 INFO (jetty-launcher-252-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 176127 INFO (jetty-launcher-252-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3cedbcb0{/solr,null,AVAILABLE}
[junit4] 2> 176130 INFO (jetty-launcher-252-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5d953d4{HTTP/1.1,[http/1.1]}{127.0.0.1:42810}
[junit4] 2> 176130 INFO (jetty-launcher-252-thread-1) [ ] o.e.j.s.Server Started @176185ms
[junit4] 2> 176130 INFO (jetty-launcher-252-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42810}
[junit4] 2> 176130 ERROR (jetty-launcher-252-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 176130 INFO (jetty-launcher-252-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 176130 INFO (jetty-launcher-252-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.5.0
[junit4] 2> 176131 INFO (jetty-launcher-252-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 176131 INFO (jetty-launcher-252-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 176131 INFO (jetty-launcher-252-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-28T18:59:04.736Z
[junit4] 2> 176131 INFO (jetty-launcher-252-thread-1) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 176155 INFO (jetty-launcher-252-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 176155 INFO (jetty-launcher-252-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 176155 INFO (jetty-launcher-252-thread-2) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 176166 INFO (jetty-launcher-252-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 176166 INFO (jetty-launcher-252-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 176166 INFO (jetty-launcher-252-thread-3) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 176170 INFO (jetty-launcher-252-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@106d49a{/solr,null,AVAILABLE}
[junit4] 2> 176171 INFO (jetty-launcher-252-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@388b78c{HTTP/1.1,[http/1.1]}{127.0.0.1:44746}
[junit4] 2> 176171 INFO (jetty-launcher-252-thread-3) [ ] o.e.j.s.Server Started @176226ms
[junit4] 2> 176171 INFO (jetty-launcher-252-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44746}
[junit4] 2> 176171 ERROR (jetty-launcher-252-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 176171 INFO (jetty-launcher-252-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 176171 INFO (jetty-launcher-252-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.5.0
[junit4] 2> 176171 INFO (jetty-launcher-252-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 176171 INFO (jetty-launcher-252-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 176171 INFO (jetty-launcher-252-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-28T18:59:04.776Z
[junit4] 2> 176224 INFO (zkConnectionManagerCallback-257-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 176225 INFO (jetty-launcher-252-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 176276 INFO (jetty-launcher-252-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@690f971c{/solr,null,AVAILABLE}
[junit4] 2> 176277 INFO (jetty-launcher-252-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@577afc61{HTTP/1.1,[http/1.1]}{127.0.0.1:34806}
[junit4] 2> 176277 INFO (jetty-launcher-252-thread-2) [ ] o.e.j.s.Server Started @176333ms
[junit4] 2> 176281 INFO (jetty-launcher-252-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34806}
[junit4] 2> 176282 ERROR (jetty-launcher-252-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 176282 INFO (jetty-launcher-252-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 176282 INFO (jetty-launcher-252-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.5.0
[junit4] 2> 176282 INFO (jetty-launcher-252-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 176282 INFO (jetty-launcher-252-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 176282 INFO (jetty-launcher-252-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-28T18:59:04.887Z
[junit4] 2> 176330 INFO (zkConnectionManagerCallback-259-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 176331 INFO (jetty-launcher-252-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 176405 INFO (zkConnectionManagerCallback-261-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 176422 INFO (jetty-launcher-252-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 176730 INFO (jetty-launcher-252-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44897/solr
[junit4] 2> 176798 INFO (zkConnectionManagerCallback-265-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 176854 INFO (zkConnectionManagerCallback-267-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177418 INFO (jetty-launcher-252-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44897/solr
[junit4] 2> 177458 INFO (zkConnectionManagerCallback-273-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177479 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 177480 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42810_solr
[junit4] 2> 177481 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.c.Overseer Overseer (id=74054098569920517-127.0.0.1:42810_solr-n_0000000000) starting
[junit4] 2> 177522 INFO (zkConnectionManagerCallback-275-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177534 INFO (jetty-launcher-252-thread-2) [n:127.0.0.1:34806_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 177535 INFO (jetty-launcher-252-thread-2) [n:127.0.0.1:34806_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34806_solr
[junit4] 2> 177549 INFO (zkConnectionManagerCallback-280-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177560 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 177562 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44897/solr ready
[junit4] 2> 177674 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 177674 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42810_solr
[junit4] 2> 177700 INFO (zkCallback-266-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 177700 INFO (zkCallback-274-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 177701 INFO (OverseerStateUpdate-74054098569920517-127.0.0.1:42810_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 177849 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 177853 INFO (zkConnectionManagerCallback-287-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177855 INFO (jetty-launcher-252-thread-2) [n:127.0.0.1:34806_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 177856 INFO (jetty-launcher-252-thread-2) [n:127.0.0.1:34806_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44897/solr ready
[junit4] 2> 177898 INFO (jetty-launcher-252-thread-2) [n:127.0.0.1:34806_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 177916 INFO (jetty-launcher-252-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44897/solr
[junit4] 2> 177964 INFO (zkCallback-279-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 177982 INFO (zkConnectionManagerCallback-293-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 178008 INFO (zkConnectionManagerCallback-295-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 178138 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42810.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 178140 INFO (jetty-launcher-252-thread-2) [n:127.0.0.1:34806_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34806.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 178183 INFO (jetty-launcher-252-thread-2) [n:127.0.0.1:34806_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34806.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 178184 INFO (jetty-launcher-252-thread-2) [n:127.0.0.1:34806_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34806.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 178188 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42810.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 178188 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42810.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 178189 INFO (jetty-launcher-252-thread-1) [n:127.0.0.1:42810_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/.
[junit4] 2> 178194 INFO (jetty-launcher-252-thread-2) [n:127.0.0.1:34806_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/.
[junit4] 2> 178405 INFO (jetty-launcher-252-thread-3) [n:127.0.0.1:44746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 178414 INFO (jetty-launcher-252-thread-3) [n:127.0.0.1:44746_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 178424 INFO (jetty-launcher-252-thread-3) [n:127.0.0.1:44746_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 178424 INFO (jetty-launcher-252-thread-3) [n:127.0.0.1:44746_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44746_solr
[junit4] 2> 178426 INFO (zkCallback-279-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 178426 INFO (zkCallback-274-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 178427 INFO (zkCallback-266-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 178462 INFO (zkCallback-286-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 178521 INFO (zkCallback-294-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 178542 INFO (zkConnectionManagerCallback-302-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 178543 INFO (jetty-launcher-252-thread-3) [n:127.0.0.1:44746_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 178545 INFO (jetty-launcher-252-thread-3) [n:127.0.0.1:44746_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44897/solr ready
[junit4] 2> 178570 INFO (jetty-launcher-252-thread-3) [n:127.0.0.1:44746_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 178593 INFO (jetty-launcher-252-thread-3) [n:127.0.0.1:44746_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44746.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 178795 INFO (jetty-launcher-252-thread-3) [n:127.0.0.1:44746_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44746.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 178796 INFO (jetty-launcher-252-thread-3) [n:127.0.0.1:44746_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44746.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 178802 INFO (jetty-launcher-252-thread-3) [n:127.0.0.1:44746_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/.
[junit4] 2> 178918 INFO (zkConnectionManagerCallback-305-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 178932 INFO (zkConnectionManagerCallback-310-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 178935 INFO (SUITE-LIROnShardRestartTest-seed#[B342DA41149789B6]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 178936 INFO (SUITE-LIROnShardRestartTest-seed#[B342DA41149789B6]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44897/solr ready
[junit4] 2> 178967 INFO (TEST-LIROnShardRestartTest.testAllReplicasInLIR-seed#[B342DA41149789B6]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAllReplicasInLIR
[junit4] 2> 179012 INFO (qtp671379812-829) [n:127.0.0.1:42810_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&name=allReplicasInLIR&nrtReplicas=3&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 179017 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection allReplicasInLIR
[junit4] 2> 179020 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/allReplicasInLIR.AUTOCREATED/managed-schema
[junit4] 2> 179030 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/allReplicasInLIR.AUTOCREATED/protwords.txt
[junit4] 2> 179032 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/allReplicasInLIR.AUTOCREATED/solrconfig.xml
[junit4] 2> 179036 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/allReplicasInLIR.AUTOCREATED/synonyms.txt
[junit4] 2> 179038 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/allReplicasInLIR.AUTOCREATED/stopwords.txt
[junit4] 2> 179041 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/contractions_it.txt
[junit4] 2> 179044 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/contractions_ca.txt
[junit4] 2> 179046 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stemdict_nl.txt
[junit4] 2> 179051 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_no.txt
[junit4] 2> 179053 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_hy.txt
[junit4] 2> 179060 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_id.txt
[junit4] 2> 179062 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_hi.txt
[junit4] 2> 179072 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ga.txt
[junit4] 2> 179074 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_da.txt
[junit4] 2> 179083 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ja.txt
[junit4] 2> 179085 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_pt.txt
[junit4] 2> 179087 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_hu.txt
[junit4] 2> 179089 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_el.txt
[junit4] 2> 179091 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ru.txt
[junit4] 2> 179092 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_tr.txt
[junit4] 2> 179103 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ar.txt
[junit4] 2> 179105 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/userdict_ja.txt
[junit4] 2> 179203 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_eu.txt
[junit4] 2> 179205 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_it.txt
[junit4] 2> 179207 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_cz.txt
[junit4] 2> 179209 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stoptags_ja.txt
[junit4] 2> 179211 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/contractions_fr.txt
[junit4] 2> 179213 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_de.txt
[junit4] 2> 179215 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_fa.txt
[junit4] 2> 179226 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/contractions_ga.txt
[junit4] 2> 179228 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ca.txt
[junit4] 2> 179230 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_nl.txt
[junit4] 2> 179232 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_bg.txt
[junit4] 2> 179234 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_en.txt
[junit4] 2> 179236 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_es.txt
[junit4] 2> 179238 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_lv.txt
[junit4] 2> 179240 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_ro.txt
[junit4] 2> 179242 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_th.txt
[junit4] 2> 179243 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_fr.txt
[junit4] 2> 179245 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_sv.txt
[junit4] 2> 179251 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_fi.txt
[junit4] 2> 179253 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/hyphenations_ga.txt
[junit4] 2> 179255 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/allReplicasInLIR.AUTOCREATED/lang/stopwords_gl.txt
[junit4] 2> 179257 INFO (OverseerThreadFactory-378-thread-1) [ ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/allReplicasInLIR.AUTOCREATED/params.json
[junit4] 2> 179382 INFO (OverseerStateUpdate-74054098569920517-127.0.0.1:42810_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"allReplicasInLIR",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"allReplicasInLIR_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42810/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 179388 INFO (OverseerStateUpdate-74054098569920517-127.0.0.1:42810_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"allReplicasInLIR",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"allReplicasInLIR_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44746/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 179392 INFO (OverseerStateUpdate-74054098569920517-127.0.0.1:42810_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"allReplicasInLIR",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"allReplicasInLIR_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34806/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 179730 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr x:allReplicasInLIR_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=allReplicasInLIR.AUTOCREATED&newCollection=true&name=allReplicasInLIR_shard1_replica_n2&action=CREATE&numShards=1&collection=allReplicasInLIR&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 179738 INFO (qtp671379812-822) [n:127.0.0.1:42810_solr x:allReplicasInLIR_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=allReplicasInLIR.AUTOCREATED&newCollection=true&name=allReplicasInLIR_shard1_replica_n1&action=CREATE&numShards=1&collection=allReplicasInLIR&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 179755 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr x:allReplicasInLIR_shard1_replica_n4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=allReplicasInLIR.AUTOCREATED&newCollection=true&name=allReplicasInLIR_shard1_replica_n4&action=CREATE&numShards=1&collection=allReplicasInLIR&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 179756 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 180835 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 180837 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 180848 INFO (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 180849 INFO (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 180875 WARN (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../contrib/extraction/lib
[junit4] 2> 180875 WARN (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../dist
[junit4] 2> 180875 WARN (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../contrib/clustering/lib
[junit4] 2> 180876 WARN (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../dist
[junit4] 2> 180876 WARN (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../contrib/langid/lib
[junit4] 2> 180876 WARN (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../dist
[junit4] 2> 180876 WARN (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../contrib/velocity/lib
[junit4] 2> 180877 WARN (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1/../../../../dist
[junit4] 2> 180881 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 180881 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 180881 WARN (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../contrib/extraction/lib
[junit4] 2> 180900 WARN (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../dist
[junit4] 2> 180900 WARN (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../contrib/clustering/lib
[junit4] 2> 180901 WARN (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../dist
[junit4] 2> 180901 WARN (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../contrib/langid/lib
[junit4] 2> 180901 WARN (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../dist
[junit4] 2> 180902 WARN (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../contrib/velocity/lib
[junit4] 2> 180902 WARN (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4/../../../../dist
[junit4] 2> 180922 WARN (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../contrib/extraction/lib
[junit4] 2> 180922 WARN (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../dist
[junit4] 2> 180922 WARN (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../contrib/clustering/lib
[junit4] 2> 180923 WARN (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../dist
[junit4] 2> 180923 WARN (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../contrib/langid/lib
[junit4] 2> 180923 WARN (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../dist
[junit4] 2> 180923 WARN (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../contrib/velocity/lib
[junit4] 2> 180924 WARN (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2/../../../../dist
[junit4] 2> 180968 INFO (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 180978 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 180987 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 181332 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.s.IndexSchema [allReplicasInLIR_shard1_replica_n2] Schema name=default-config
[junit4] 2> 181385 INFO (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.s.IndexSchema [allReplicasInLIR_shard1_replica_n1] Schema name=default-config
[junit4] 2> 181392 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.s.IndexSchema [allReplicasInLIR_shard1_replica_n4] Schema name=default-config
[junit4] 2> 183299 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
[junit4] 2> 183323 INFO (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
[junit4] 2> 183503 INFO (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'allReplicasInLIR_shard1_replica_n1' using configuration from collection allReplicasInLIR, trusted=true
[junit4] 2> 183504 INFO (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42810.solr.core.allReplicasInLIR.shard1.replica_n1' (registry 'solr.core.allReplicasInLIR.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 183512 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
[junit4] 2> 183525 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'allReplicasInLIR_shard1_replica_n2' using configuration from collection allReplicasInLIR, trusted=true
[junit4] 2> 183525 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44746.solr.core.allReplicasInLIR.shard1.replica_n2' (registry 'solr.core.allReplicasInLIR.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 183680 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'allReplicasInLIR_shard1_replica_n4' using configuration from collection allReplicasInLIR, trusted=true
[junit4] 2> 183683 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34806.solr.core.allReplicasInLIR.shard1.replica_n4' (registry 'solr.core.allReplicasInLIR.shard1.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@49876805
[junit4] 2> 183755 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 183755 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.c.SolrCore [[allReplicasInLIR_shard1_replica_n4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/allReplicasInLIR_shard1_replica_n4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node2/./allReplicasInLIR_shard1_replica_n4/data/]
[junit4] 2> 183759 INFO (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 183759 INFO (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.c.SolrCore [[allReplicasInLIR_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/allReplicasInLIR_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node1/./allReplicasInLIR_shard1_replica_n1/data/]
[junit4] 2> 183782 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 183782 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.c.SolrCore [[allReplicasInLIR_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/allReplicasInLIR_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.LIROnShardRestartTest_B342DA41149789B6-001/tempDir-001/node3/./allReplicasInLIR_shard1_replica_n2/data/]
[junit4] 2> 184386 INFO (qtp671379812-822) [n:127.0.0.1:42810_solr c:allReplicasInLIR s:shard1 r:core_node3 x:allReplicasInLIR_shard1_replica_n1] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
[junit4] 2> 184390 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
[junit4] 2> 184406 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.r.XSLTResponseWriter xsltCacheLifetimeSeconds=5
[junit4] 2> 185622 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 185622 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 185623 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 185624 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 185628 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@220ae7ef[allReplicasInLIR_shard1_replica_n2] main]
[junit4] 2> 185632 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/allReplicasInLIR.AUTOCREATED
[junit4] 2> 185632 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/allReplicasInLIR.AUTOCREATED
[junit4] 2> 185636 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/allReplicasInLIR.AUTOCREATED/managed-schema
[junit4] 2> 185656 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 185659 INFO (qtp1307543126-840) [n:127.0.0.1:44746_solr c:allReplicasInLIR s:shard1 r:core_node5 x:allReplicasInLIR_shard1_replica_n2] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 185693 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 185694 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 185695 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: if uncommited for 15000ms;
[junit4] 2> 185695 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 185710 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@687263a3[allReplicasInLIR_shard1_replica_n4] main]
[junit4] 2> 185712 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/allReplicasInLIR.AUTOCREATED
[junit4] 2> 185713 INFO (qtp336761060-837) [n:127.0.0.1:34806_solr c:allReplicasInLIR s:shard1 r:core_node6 x:allReplicasInLIR_shard1_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/allReplicasInLIR.AUTOCREATED
[junit4] 2> 185713 INFO (qtp336761060-837) [
[...truncated too long message...]
lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null692779144
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null692779144
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null692779144
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null988032463
[copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null988032463
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null988032463
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 62ms :: artifacts dl 15ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 357 minutes 9 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-NightlyTests-7.x - Build # 249 - Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/249/
3 tests failed.
FAILED: org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings
Error Message:
finalOffset expected:<40> but was:<38>
Stack Trace:
java.lang.AssertionError: finalOffset expected:<40> but was:<38>
at __randomizedtesting.SeedInfo.seed([319F02FFE4F7A86B:5BC4BDEEBDB98898]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:305)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:320)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:324)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:860)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:659)
at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:561)
at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:893)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test
Error Message:
Expected numSlices=5 numReplicas=1 but found DocCollection(solrj_collection3//collections/solrj_collection3/state.json/25)={ "pullReplicas":"0", "replicationFactor":"1", "shards":{ "shard1":{ "range":"80000000-b332ffff", "state":"active", "replicas":{"core_node3":{ "dataDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node3/data/", "base_url":"http://127.0.0.1:43328/d_vee", "node_name":"127.0.0.1:43328_d_vee", "type":"NRT", "ulogDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node3/data/tlog", "core":"solrj_collection3_shard1_replica_n1", "shared_storage":"true", "state":"active", "leader":"true"}}}, "shard2":{ "range":"b3330000-e665ffff", "state":"active", "replicas":{"core_node5":{ "dataDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node5/data/", "base_url":"http://127.0.0.1:49955/d_vee", "node_name":"127.0.0.1:49955_d_vee", "type":"NRT", "ulogDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node5/data/tlog", "core":"solrj_collection3_shard2_replica_n2", "shared_storage":"true", "state":"active", "leader":"true"}}}, "shard3":{ "range":"e6660000-1998ffff", "state":"active", "replicas":{"core_node7":{ "dataDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node7/data/", "base_url":"http://127.0.0.1:49955/d_vee", "node_name":"127.0.0.1:49955_d_vee", "type":"NRT", "ulogDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node7/data/tlog", "core":"solrj_collection3_shard3_replica_n4", "shared_storage":"true", "state":"active", "leader":"true"}}}, "shard4":{ "range":"19990000-4ccbffff", "state":"active", "replicas":{"core_node9":{ "dataDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node9/data/", "base_url":"http://127.0.0.1:49435/d_vee", "node_name":"127.0.0.1:49435_d_vee", "type":"NRT", "ulogDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node9/data/tlog", "core":"solrj_collection3_shard4_replica_n6", "shared_storage":"true", "state":"active", "leader":"true"}}}, "shard5":{ "range":"4ccc0000-7fffffff", "state":"active", "replicas":{"core_node10":{ "core":"solrj_collection3_shard5_replica_n8", "base_url":"http://127.0.0.1:43328/d_vee", "node_name":"127.0.0.1:43328_d_vee", "state":"down", "type":"NRT", "leader":"true"}}}}, "router":{ "field":"myOwnField", "name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"true", "nrtReplicas":"1", "tlogReplicas":"0"} with /live_nodes: [127.0.0.1:43328_d_vee, 127.0.0.1:54753_d_vee, 127.0.0.1:49435_d_vee, 127.0.0.1:57980_d_vee, 127.0.0.1:49955_d_vee]
Stack Trace:
java.lang.AssertionError: Expected numSlices=5 numReplicas=1 but found DocCollection(solrj_collection3//collections/solrj_collection3/state.json/25)={
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{
"shard1":{
"range":"80000000-b332ffff",
"state":"active",
"replicas":{"core_node3":{
"dataDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node3/data/",
"base_url":"http://127.0.0.1:43328/d_vee",
"node_name":"127.0.0.1:43328_d_vee",
"type":"NRT",
"ulogDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node3/data/tlog",
"core":"solrj_collection3_shard1_replica_n1",
"shared_storage":"true",
"state":"active",
"leader":"true"}}},
"shard2":{
"range":"b3330000-e665ffff",
"state":"active",
"replicas":{"core_node5":{
"dataDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node5/data/",
"base_url":"http://127.0.0.1:49955/d_vee",
"node_name":"127.0.0.1:49955_d_vee",
"type":"NRT",
"ulogDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node5/data/tlog",
"core":"solrj_collection3_shard2_replica_n2",
"shared_storage":"true",
"state":"active",
"leader":"true"}}},
"shard3":{
"range":"e6660000-1998ffff",
"state":"active",
"replicas":{"core_node7":{
"dataDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node7/data/",
"base_url":"http://127.0.0.1:49955/d_vee",
"node_name":"127.0.0.1:49955_d_vee",
"type":"NRT",
"ulogDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node7/data/tlog",
"core":"solrj_collection3_shard3_replica_n4",
"shared_storage":"true",
"state":"active",
"leader":"true"}}},
"shard4":{
"range":"19990000-4ccbffff",
"state":"active",
"replicas":{"core_node9":{
"dataDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node9/data/",
"base_url":"http://127.0.0.1:49435/d_vee",
"node_name":"127.0.0.1:49435_d_vee",
"type":"NRT",
"ulogDir":"hdfs://localhost:53444/solr_hdfs_home/solrj_collection3/core_node9/data/tlog",
"core":"solrj_collection3_shard4_replica_n6",
"shared_storage":"true",
"state":"active",
"leader":"true"}}},
"shard5":{
"range":"4ccc0000-7fffffff",
"state":"active",
"replicas":{"core_node10":{
"core":"solrj_collection3_shard5_replica_n8",
"base_url":"http://127.0.0.1:43328/d_vee",
"node_name":"127.0.0.1:43328_d_vee",
"state":"down",
"type":"NRT",
"leader":"true"}}}},
"router":{
"field":"myOwnField",
"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"true",
"nrtReplicas":"1",
"tlogReplicas":"0"} with /live_nodes: [127.0.0.1:43328_d_vee, 127.0.0.1:54753_d_vee, 127.0.0.1:49435_d_vee, 127.0.0.1:57980_d_vee, 127.0.0.1:49955_d_vee]
at __randomizedtesting.SeedInfo.seed([4CEC1947D1134C5F:C4B8269D7FEF21A7]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.assertSliceAndReplicaCount(SharedFSAutoReplicaFailoverTest.java:415)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.testBasics(SharedFSAutoReplicaFailoverTest.java:257)
at org.apache.solr.cloud.SharedFSAutoReplicaFailoverTest.test(SharedFSAutoReplicaFailoverTest.java:145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:1008)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:983)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest.testBiDir
Error Message:
Captured an uncaught exception in thread: Thread[id=11611, name=cdcr-replicator-2995-thread-1, state=RUNNABLE, group=TGRP-CdcrBidirectionalTest]
Stack Trace:
com.carrotsearch.randomizedtesting.UncaughtExceptionError: Captured an uncaught exception in thread: Thread[id=11611, name=cdcr-replicator-2995-thread-1, state=RUNNABLE, group=TGRP-CdcrBidirectionalTest]
Caused by: java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([4CEC1947D1134C5F]:0)
at org.apache.solr.update.CdcrUpdateLog$CdcrLogReader.forwardSeek(CdcrUpdateLog.java:611)
at org.apache.solr.handler.CdcrReplicator.run(CdcrReplicator.java:125)
at org.apache.solr.handler.CdcrReplicatorScheduler.lambda$null$0(CdcrReplicatorScheduler.java:81)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:209)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 3832 lines...]
[junit4] Suite: org.apache.lucene.analysis.core.TestRandomChains
[junit4] 2> TEST FAIL: useCharFilter=false text='c \u052c\u8443\ueedd\ua291x\ue16d\ud87b\ude42\u7e7f jnssft \u1df5\u1df3\u1dcc\u1dc8\u1ddf\u1ddf\u1dc2\u1de8\u1dde\u1de1 t mgzelt ?'
[junit4] 2> Exception from random analyzer:
[junit4] 2> charfilters=
[junit4] 2> org.apache.lucene.analysis.charfilter.MappingCharFilter(org.apache.lucene.analysis.charfilter.NormalizeCharMap@1490adc8, java.io.StringReader@6d79ac4d)
[junit4] 2> org.apache.lucene.analysis.charfilter.HTMLStripCharFilter(org.apache.lucene.analysis.charfilter.MappingCharFilter@5f9d7e8f)
[junit4] 2> tokenizer=
[junit4] 2> org.apache.lucene.analysis.standard.UAX29URLEmailTokenizer(org.apache.lucene.util.AttributeFactory$1@7cd0d664)
[junit4] 2> filters=
[junit4] 2> Conditional:org.apache.lucene.analysis.no.NorwegianLightStemFilter(OneTimeWrapper@2904482 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,flags=0,payload=null,keyword=false)
[junit4] 2> Conditional:org.apache.lucene.analysis.shingle.FixedShingleFilter(OneTimeWrapper@6b50d367 term=,bytes=[],startOffset=0,endOffset=0,positionIncrement=1,positionLength=1,type=word,termFrequency=1,flags=0,payload=null,keyword=false, 3, <KATAKANA>, <HANGUL>)
[junit4] 2> NOTE: download the large Jenkins line-docs file by running 'ant get-jenkins-line-docs' in the lucene directory.
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestRandomChains -Dtests.method=testRandomChainsWithLargeStrings -Dtests.seed=319F02FFE4F7A86B -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=he-IL -Dtests.timezone=Pacific/Gambier -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 16.0s J0 | TestRandomChains.testRandomChainsWithLargeStrings <<<
[junit4] > Throwable #1: java.lang.AssertionError: finalOffset expected:<40> but was:<38>
[junit4] > at __randomizedtesting.SeedInfo.seed([319F02FFE4F7A86B:5BC4BDEEBDB98898]:0)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:305)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:320)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.assertTokenStreamContents(BaseTokenStreamTestCase.java:324)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkAnalysisConsistency(BaseTokenStreamTestCase.java:860)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:659)
[junit4] > at org.apache.lucene.analysis.BaseTokenStreamTestCase.checkRandomData(BaseTokenStreamTestCase.java:561)
[junit4] > at org.apache.lucene.analysis.core.TestRandomChains.testRandomChainsWithLargeStrings(TestRandomChains.java:893)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/build/analysis/common/test/J0/temp/lucene.analysis.core.TestRandomChains_319F02FFE4F7A86B-001
[junit4] 2> NOTE: test params are: codec=SimpleText, sim=RandomSimilarity(queryNorm=true): {dummy=DFR GB1}, locale=he-IL, timezone=Pacific/Gambier
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=269319248,total=336592896
[junit4] 2> NOTE: All tests run in this JVM: [TestArabicStemFilter, TestHindiAnalyzer, TestGermanLightStemFilterFactory, TestPortugueseMinimalStemFilterFactory, DateRecognizerFilterFactoryTest, TestFinnishLightStemFilter, TestCommonGramsQueryFilterFactory, TestNGramFilters, TestHyphenationCompoundWordTokenFilterFactory, TestBugInSomething, TestSpanishAnalyzer, TestNorwegianMinimalStemFilterFactory, TestLimitTokenPositionFilter, TestCondition, TestKeywordMarkerFilterFactory, TestPatternTokenizerFactory, EdgeNGramTokenizerTest, TestTurkishAnalyzer, TestSwedishLightStemFilter, TestPatternCaptureGroupTokenFilter, TestSolrSynonymParser, TestCzechAnalyzer, TestGalicianMinimalStemFilterFactory, TestCJKWidthFilter, TestGermanNormalizationFilter, TestSoraniNormalizationFilter, TestElision, TestRussianLightStemFilterFactory, TestFrenchMinimalStemFilterFactory, TestRomanianAnalyzer, TestBasqueAnalyzer, TestLengthFilterFactory, TestLithuanianAnalyzer, ShingleFilterTest, TestSpaces, TestConditionalTokenFilter, QueryAutoStopWordAnalyzerTest, TestWordDelimiterFilter, TestHTMLStripCharFilterFactory, WikipediaTokenizerTest, TestRollingCharBuffer, TestNeedAffix, TestTwoFold, TokenOffsetPayloadTokenFilterTest, TestGreekStemFilterFactory, TestGermanStemFilterFactory, TestStemmerOverrideFilterFactory, TestThaiTokenizerFactory, TestCJKAnalyzer, TestCJKBigramFilterFactory, TestSoraniStemFilterFactory, CommonGramsFilterTest, TestDuelingAnalyzers, TestFlattenGraphFilter, TestKeywordAnalyzer, TestKeywordTokenizer, TestRandomChains]
[junit4] Completed [291/291 (1!)] on J0 in 225.76s, 2 tests, 1 failure <<< FAILURES!
[...truncated 10093 lines...]
[junit4] Suite: org.apache.solr.cloud.cdcr.CdcrBidirectionalTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/init-core-data-001
[junit4] 2> 1623480 WARN (SUITE-CdcrBidirectionalTest-seed#[4CEC1947D1134C5F]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=5 numCloses=5
[junit4] 2> 1623481 INFO (SUITE-CdcrBidirectionalTest-seed#[4CEC1947D1134C5F]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1623482 INFO (SUITE-CdcrBidirectionalTest-seed#[4CEC1947D1134C5F]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1623482 INFO (SUITE-CdcrBidirectionalTest-seed#[4CEC1947D1134C5F]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1623484 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBiDir
[junit4] 2> 1623485 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster2-001
[junit4] 2> 1623485 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1623486 INFO (Thread-4710) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1623486 INFO (Thread-4710) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1623488 ERROR (Thread-4710) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1623595 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.ZkTestServer start zk server on port:53722
[junit4] 2> 1623598 INFO (zkConnectionManagerCallback-2895-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1623612 INFO (jetty-launcher-2892-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 1623619 INFO (jetty-launcher-2892-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1623619 INFO (jetty-launcher-2892-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1623620 INFO (jetty-launcher-2892-thread-1) [ ] o.e.j.s.session node0 Scavenging every 660000ms
[junit4] 2> 1623620 INFO (jetty-launcher-2892-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1608dc09{/solr,null,AVAILABLE}
[junit4] 2> 1623621 INFO (jetty-launcher-2892-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2636971f{HTTP/1.1,[http/1.1]}{127.0.0.1:52472}
[junit4] 2> 1623621 INFO (jetty-launcher-2892-thread-1) [ ] o.e.j.s.Server Started @1623680ms
[junit4] 2> 1623621 INFO (jetty-launcher-2892-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52472}
[junit4] 2> 1623621 ERROR (jetty-launcher-2892-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1623621 INFO (jetty-launcher-2892-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1623621 INFO (jetty-launcher-2892-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.5.0
[junit4] 2> 1623621 INFO (jetty-launcher-2892-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1623621 INFO (jetty-launcher-2892-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1623621 INFO (jetty-launcher-2892-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-27T15:33:01.368Z
[junit4] 2> 1623639 INFO (zkConnectionManagerCallback-2897-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1623640 INFO (jetty-launcher-2892-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1624032 INFO (jetty-launcher-2892-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:53722/solr
[junit4] 2> 1624033 INFO (zkConnectionManagerCallback-2901-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1624034 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10711ec6d830002, likely client has closed socket
[junit4] 2> 1624036 INFO (zkConnectionManagerCallback-2903-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1624356 INFO (jetty-launcher-2892-thread-1) [n:127.0.0.1:52472_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1624357 INFO (jetty-launcher-2892-thread-1) [n:127.0.0.1:52472_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:52472_solr
[junit4] 2> 1624358 INFO (jetty-launcher-2892-thread-1) [n:127.0.0.1:52472_solr ] o.a.s.c.Overseer Overseer (id=74047626022158339-127.0.0.1:52472_solr-n_0000000000) starting
[junit4] 2> 1624376 INFO (zkConnectionManagerCallback-2910-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1624384 INFO (jetty-launcher-2892-thread-1) [n:127.0.0.1:52472_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53722/solr ready
[junit4] 2> 1624394 INFO (jetty-launcher-2892-thread-1) [n:127.0.0.1:52472_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52472_solr
[junit4] 2> 1624402 INFO (OverseerStateUpdate-74047626022158339-127.0.0.1:52472_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1624438 INFO (zkCallback-2909-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1624446 INFO (jetty-launcher-2892-thread-1) [n:127.0.0.1:52472_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1624484 INFO (jetty-launcher-2892-thread-1) [n:127.0.0.1:52472_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52472.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b517a1f
[junit4] 2> 1624497 INFO (jetty-launcher-2892-thread-1) [n:127.0.0.1:52472_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52472.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b517a1f
[junit4] 2> 1624497 INFO (jetty-launcher-2892-thread-1) [n:127.0.0.1:52472_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52472.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b517a1f
[junit4] 2> 1624499 INFO (jetty-launcher-2892-thread-1) [n:127.0.0.1:52472_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster2-001/node1/.
[junit4] 2> 1624543 INFO (zkConnectionManagerCallback-2913-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1624547 INFO (zkConnectionManagerCallback-2916-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1624550 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster1-001
[junit4] 2> 1624550 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1624551 INFO (Thread-4720) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1624551 INFO (Thread-4720) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1624554 ERROR (Thread-4720) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1624650 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.ZkTestServer start zk server on port:43524
[junit4] 2> 1624653 INFO (zkConnectionManagerCallback-2920-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1624658 INFO (jetty-launcher-2917-thread-1) [ ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 1.8.0_172-b11
[junit4] 2> 1624660 INFO (jetty-launcher-2917-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1624660 INFO (jetty-launcher-2917-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1624660 INFO (jetty-launcher-2917-thread-1) [ ] o.e.j.s.session node0 Scavenging every 600000ms
[junit4] 2> 1624663 INFO (jetty-launcher-2917-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@24cd6e53{/solr,null,AVAILABLE}
[junit4] 2> 1624663 INFO (jetty-launcher-2917-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5efdedee{HTTP/1.1,[http/1.1]}{127.0.0.1:43249}
[junit4] 2> 1624663 INFO (jetty-launcher-2917-thread-1) [ ] o.e.j.s.Server Started @1624723ms
[junit4] 2> 1624663 INFO (jetty-launcher-2917-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43249}
[junit4] 2> 1624664 ERROR (jetty-launcher-2917-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1624664 INFO (jetty-launcher-2917-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 1624664 INFO (jetty-launcher-2917-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.5.0
[junit4] 2> 1624664 INFO (jetty-launcher-2917-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1624664 INFO (jetty-launcher-2917-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1624664 INFO (jetty-launcher-2917-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-06-27T15:33:02.411Z
[junit4] 2> 1624673 INFO (zkConnectionManagerCallback-2922-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1624673 INFO (jetty-launcher-2917-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1625026 INFO (jetty-launcher-2917-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43524/solr
[junit4] 2> 1625032 INFO (zkConnectionManagerCallback-2926-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1625035 INFO (zkConnectionManagerCallback-2928-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1625295 INFO (jetty-launcher-2917-thread-1) [n:127.0.0.1:43249_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1625296 INFO (jetty-launcher-2917-thread-1) [n:127.0.0.1:43249_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43249_solr
[junit4] 2> 1625298 INFO (jetty-launcher-2917-thread-1) [n:127.0.0.1:43249_solr ] o.a.s.c.Overseer Overseer (id=74047626092019715-127.0.0.1:43249_solr-n_0000000000) starting
[junit4] 2> 1625312 INFO (zkConnectionManagerCallback-2935-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1625319 INFO (jetty-launcher-2917-thread-1) [n:127.0.0.1:43249_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43524/solr ready
[junit4] 2> 1625323 INFO (jetty-launcher-2917-thread-1) [n:127.0.0.1:43249_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43249_solr
[junit4] 2> 1625352 INFO (zkCallback-2934-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1625353 INFO (zkCallback-2927-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1625357 INFO (jetty-launcher-2917-thread-1) [n:127.0.0.1:43249_solr ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
[junit4] 2> 1625384 INFO (jetty-launcher-2917-thread-1) [n:127.0.0.1:43249_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43249.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b517a1f
[junit4] 2> 1625395 INFO (jetty-launcher-2917-thread-1) [n:127.0.0.1:43249_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43249.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b517a1f
[junit4] 2> 1625396 INFO (jetty-launcher-2917-thread-1) [n:127.0.0.1:43249_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43249.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b517a1f
[junit4] 2> 1625398 INFO (jetty-launcher-2917-thread-1) [n:127.0.0.1:43249_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster1-001/node1/.
[junit4] 2> 1625443 INFO (zkConnectionManagerCallback-2938-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1625449 INFO (zkConnectionManagerCallback-2941-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1625450 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.c.CdcrBidirectionalTest cluster2 zkHost = 127.0.0.1:53722/solr
[junit4] 2> 1625451 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.c.CdcrBidirectionalTest cluster1 zkHost = 127.0.0.1:43524/solr
[junit4] 2> 1625456 INFO (zkConnectionManagerCallback-2943-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1625474 INFO (zkConnectionManagerCallback-2947-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1625476 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1625478 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43524/solr ready
[junit4] 2> 1625479 INFO (qtp800323450-11452) [n:127.0.0.1:43249_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=cdcr-cluster1&maxShardsPerNode=2&name=cdcr-cluster1&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1625484 INFO (OverseerThreadFactory-2956-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection cdcr-cluster1
[junit4] 2> 1625741 INFO (OverseerStateUpdate-74047626092019715-127.0.0.1:43249_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-cluster1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43249/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1625744 INFO (OverseerStateUpdate-74047626092019715-127.0.0.1:43249_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"cdcr-cluster1_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43249/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1625950 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster1_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 1625951 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1625951 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 1626979 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 1626987 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 1626993 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.IndexSchema [cdcr-cluster1_shard2_replica_n2] Schema name=minimal
[junit4] 2> 1626997 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1627041 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster1_shard2_replica_n2' using configuration from collection cdcr-cluster1, trusted=true
[junit4] 2> 1627041 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43249.solr.core.cdcr-cluster1.shard2.replica_n2' (registry 'solr.core.cdcr-cluster1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b517a1f
[junit4] 2> 1627042 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1627042 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore [[cdcr-cluster1_shard2_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster1-001/node1/cdcr-cluster1_shard2_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster1-001/node1/./cdcr-cluster1_shard2_replica_n2/data/]
[junit4] 2> 1627051 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-cluster1_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1627057 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1627057 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster1_shard1_replica_n1' using configuration from collection cdcr-cluster1, trusted=true
[junit4] 2> 1627058 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43249.solr.core.cdcr-cluster1.shard1.replica_n1' (registry 'solr.core.cdcr-cluster1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b517a1f
[junit4] 2> 1627058 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1627058 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-cluster1_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster1-001/node1/cdcr-cluster1_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster1-001/node1/./cdcr-cluster1_shard1_replica_n1/data/]
[junit4] 2> 1627162 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1627162 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1627164 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1627164 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1627167 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5686f0fc[cdcr-cluster1_shard1_replica_n1] main]
[junit4] 2> 1627169 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster1
[junit4] 2> 1627169 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster1
[junit4] 2> 1627169 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster1/managed-schema
[junit4] 2> 1627170 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1627171 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1627174 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-cluster1/cdcr/state/buffer
[junit4] 2> 1627175 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-cluster1/cdcr/state/process
[junit4] 2> 1627178 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604440382426316800
[junit4] 2> 1627180 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1627180 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1627180 INFO (searcherExecutor-2962-thread-1-processing-n:127.0.0.1:43249_solr x:cdcr-cluster1_shard1_replica_n1 c:cdcr-cluster1 s:shard1 r:core_node3) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-cluster1_shard1_replica_n1] Registered new searcher Searcher@5686f0fc[cdcr-cluster1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1627181 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1627181 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1627184 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7640df71[cdcr-cluster1_shard2_replica_n2] main]
[junit4] 2> 1627185 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard1 to Terms{values={core_node3=0}, version=0}
[junit4] 2> 1627198 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster1
[junit4] 2> 1627199 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster1
[junit4] 2> 1627199 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster1/managed-schema
[junit4] 2> 1627199 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1627200 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1627200 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1627200 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43249/solr/cdcr-cluster1_shard1_replica_n1/
[junit4] 2> 1627200 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1627200 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1627201 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:43249/solr/cdcr-cluster1_shard1_replica_n1/ has no replicas
[junit4] 2> 1627201 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1627290 INFO (zkCallback-2927-thread-2) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster1:shard1
[junit4] 2> 1627300 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43249/solr/cdcr-cluster1_shard1_replica_n1/ shard1
[junit4] 2> 1627301 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604440382555291648
[junit4] 2> 1627304 INFO (searcherExecutor-2961-thread-1-processing-n:127.0.0.1:43249_solr x:cdcr-cluster1_shard2_replica_n2 c:cdcr-cluster1 s:shard2 r:core_node4) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SolrCore [cdcr-cluster1_shard2_replica_n2] Registered new searcher Searcher@7640df71[cdcr-cluster1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1627314 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster1/terms/shard2 to Terms{values={core_node4=0}, version=0}
[junit4] 2> 1627319 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1627319 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1627319 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43249/solr/cdcr-cluster1_shard2_replica_n2/
[junit4] 2> 1627319 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1627320 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.SyncStrategy http://127.0.0.1:43249/solr/cdcr-cluster1_shard2_replica_n2/ has no replicas
[junit4] 2> 1627320 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1627332 INFO (zkCallback-2927-thread-2) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster1:shard2
[junit4] 2> 1627339 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43249/solr/cdcr-cluster1_shard2_replica_n2/ shard2
[junit4] 2> 1627444 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1627447 INFO (qtp800323450-11448) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster1_shard2_replica_n2&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1497
[junit4] 2> 1627452 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1627454 INFO (qtp800323450-11447) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster1&newCollection=true&collection=cdcr-cluster1&version=2&replicaType=NRT&coreNodeName=core_node3&name=cdcr-cluster1_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1503
[junit4] 2> 1627467 INFO (qtp800323450-11452) [n:127.0.0.1:43249_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1627492 INFO (OverseerCollectionConfigSetProcessor-74047626092019715-127.0.0.1:43249_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1627556 INFO (zkCallback-2927-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/state.json] for collection [cdcr-cluster1] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1628467 INFO (qtp800323450-11452) [n:127.0.0.1:43249_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=cdcr-cluster1&maxShardsPerNode=2&name=cdcr-cluster1&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2987
[junit4] 2> 1628469 INFO (zkConnectionManagerCallback-2951-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1628477 INFO (zkConnectionManagerCallback-2955-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1628479 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1628481 INFO (TEST-CdcrBidirectionalTest.testBiDir-seed#[4CEC1947D1134C5F]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53722/solr ready
[junit4] 2> 1628485 INFO (qtp609129268-11398) [n:127.0.0.1:52472_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params collection.configName=cdcr-cluster2&maxShardsPerNode=2&name=cdcr-cluster2&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1628491 INFO (OverseerThreadFactory-2943-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection cdcr-cluster2
[junit4] 2> 1628668 INFO (OverseerStateUpdate-74047626022158339-127.0.0.1:52472_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-cluster2_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:52472/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1628672 INFO (OverseerStateUpdate-74047626022158339-127.0.0.1:52472_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-cluster2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"cdcr-cluster2_shard2_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:52472/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1628880 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node2&name=cdcr-cluster2_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 1628880 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1628896 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr x:cdcr-cluster2_shard2_replica_n3] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster2_shard2_replica_n3&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 1629923 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 1629928 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.5.0
[junit4] 2> 1629947 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.s.IndexSchema [cdcr-cluster2_shard2_replica_n3] Schema name=minimal
[junit4] 2> 1629950 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1629950 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster2_shard2_replica_n3' using configuration from collection cdcr-cluster2, trusted=true
[junit4] 2> 1629953 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-cluster2_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1629955 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1629955 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-cluster2_shard1_replica_n1' using configuration from collection cdcr-cluster2, trusted=true
[junit4] 2> 1629956 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52472.solr.core.cdcr-cluster2.shard1.replica_n1' (registry 'solr.core.cdcr-cluster2.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b517a1f
[junit4] 2> 1629956 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1629956 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-cluster2_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster2-001/node1/cdcr-cluster2_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster2-001/node1/./cdcr-cluster2_shard1_replica_n1/data/]
[junit4] 2> 1629974 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52472.solr.core.cdcr-cluster2.shard2.replica_n3' (registry 'solr.core.cdcr-cluster2.shard2.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b517a1f
[junit4] 2> 1629975 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1629975 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.SolrCore [[cdcr-cluster2_shard2_replica_n3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster2-001/node1/cdcr-cluster2_shard2_replica_n3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/build/solr-core/test/J0/temp/solr.cloud.cdcr.CdcrBidirectionalTest_4CEC1947D1134C5F-001/cdcr-cluster2-001/node1/./cdcr-cluster2_shard2_replica_n3/data/]
[junit4] 2> 1630400 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1630400 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1630406 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1630406 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1630454 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1630454 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1630454 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1630454 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1630464 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1438c326[cdcr-cluster2_shard1_replica_n1] main]
[junit4] 2> 1630468 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster2
[junit4] 2> 1630471 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster2
[junit4] 2> 1630471 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster2/managed-schema
[junit4] 2> 1630472 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1630473 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1630477 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-cluster2/cdcr/state/buffer
[junit4] 2> 1630478 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-cluster2/cdcr/state/process
[junit4] 2> 1630480 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@33c4f0de[cdcr-cluster2_shard2_replica_n3] main]
[junit4] 2> 1630481 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-cluster2
[junit4] 2> 1630482 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-cluster2
[junit4] 2> 1630482 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-cluster2/managed-schema
[junit4] 2> 1630483 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1630483 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1630493 INFO (searcherExecutor-2973-thread-1-processing-n:127.0.0.1:52472_solr x:cdcr-cluster2_shard2_replica_n3 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.SolrCore [cdcr-cluster2_shard2_replica_n3] Registered new searcher Searcher@33c4f0de[cdcr-cluster2_shard2_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1630497 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604440385906540544
[junit4] 2> 1630516 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1604440385926463488
[junit4] 2> 1630517 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster2/terms/shard2 to Terms{values={core_node4=0}, version=0}
[junit4] 2> 1630518 INFO (searcherExecutor-2974-thread-1-processing-n:127.0.0.1:52472_solr x:cdcr-cluster2_shard1_replica_n1 c:cdcr-cluster2 s:shard1 r:core_node2) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-cluster2_shard1_replica_n1] Registered new searcher Searcher@1438c326[cdcr-cluster2_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1630522 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1630522 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1630522 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52472/solr/cdcr-cluster2_shard2_replica_n3/
[junit4] 2> 1630523 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1630523 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.SyncStrategy http://127.0.0.1:52472/solr/cdcr-cluster2_shard2_replica_n3/ has no replicas
[junit4] 2> 1630523 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/cdcr-cluster2/terms/shard1 to Terms{values={core_node2=0}, version=0}
[junit4] 2> 1630523 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1630528 INFO (zkCallback-2902-thread-1) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster2:shard2
[junit4] 2> 1630531 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1630531 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52472/solr/cdcr-cluster2_shard2_replica_n3/ shard2
[junit4] 2> 1630531 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1630531 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:52472/solr/cdcr-cluster2_shard1_replica_n1/
[junit4] 2> 1630532 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1630533 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:52472/solr/cdcr-cluster2_shard1_replica_n1/ has no replicas
[junit4] 2> 1630533 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1630535 INFO (zkCallback-2902-thread-1) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-cluster2:shard1
[junit4] 2> 1630549 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:52472/solr/cdcr-cluster2_shard1_replica_n1/ shard1
[junit4] 2> 1630692 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1630694 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1630694 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node2&name=cdcr-cluster2_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1814
[junit4] 2> 1630696 INFO (qtp609129268-11400) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-cluster2&newCollection=true&collection=cdcr-cluster2&version=2&replicaType=NRT&coreNodeName=core_node4&name=cdcr-cluster2_shard2_replica_n3&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1800
[junit4] 2> 1630699 INFO (qtp609129268-11398) [n:127.0.0.1:52472_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1630798 INFO (zkCallback-2902-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster2/state.json] for collection [cdcr-cluster2] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1631700 INFO (qtp609129268-11398) [n:127.0.0.1:52472_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collection.configName=cdcr-cluster2&maxShardsPerNode=2&name=cdcr-cluster2&nrtReplicas=1&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=3214
[junit4] 2> 1631713 INFO (zkConnectionManagerCallback-2962-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631716 INFO (qtp800323450-11451) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1631720 INFO (qtp800323450-11451) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53722/solr ready
[junit4] 2> 1631729 INFO (qtp609129268-11395) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1631729 INFO (qtp609129268-11395) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n3] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1631731 INFO (qtp609129268-11399) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1631731 INFO (qtp609129268-11399) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1631731 INFO (qtp609129268-11397) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n3] webapp=/solr path=/cdcr params={_stateVer_=cdcr-cluster2:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=9
[junit4] 2> 1631732 INFO (qtp800323450-11451) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-cluster2 with checkpoint -1 @ cdcr-cluster1:shard2
[junit4] 2> 1631732 INFO (qtp800323450-11451) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2, shard: shard2
[junit4] 2> 1631740 INFO (qtp800323450-11451) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
[junit4] 2> 1631742 INFO (zkCallback-2927-thread-1) [ ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/cdcr/state/process @ cdcr-cluster1:shard2
[junit4] 2> 1631742 INFO (zkCallback-2927-thread-2) [ ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-cluster1/cdcr/state/process @ cdcr-cluster1:shard1
[junit4] 2> 1631743 INFO (zkCallback-2927-thread-1) [ ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-cluster1:shard2
[junit4] 2> 1631743 INFO (qtp800323450-11451) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-cluster1:4&action=start&wt=javabin&version=2} status=0 QTime=37
[junit4] 2> 1631744 INFO (zkCallback-2927-thread-2) [ ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-cluster1:shard1
[junit4] 2> 1631745 INFO (cdcr-bootstrap-status-2958-thread-1-processing-n:127.0.0.1:43249_solr x:cdcr-cluster1_shard2_replica_n2 c:cdcr-cluster1 s:shard2 r:core_node4) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-cluster2 shard: shard2 leader: http://127.0.0.1:52472/solr/cdcr-cluster2_shard2_replica_n3/
[junit4] 2> 1631776 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n3] webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=http://127.0.0.1:43249/solr/cdcr-cluster1_shard2_replica_n2/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=30
[junit4] 2> 1631778 INFO (qtp609129268-11397) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n3] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1631779 INFO (cdcr-bootstrap-status-2958-thread-1-processing-n:127.0.0.1:43249_solr x:cdcr-cluster1_shard2_replica_n2 c:cdcr-cluster1 s:shard2 r:core_node4) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.h.CdcrReplicatorManager CDCR bootstrap running for 1 seconds, sleeping for 2000 ms
[junit4] 2> 1631780 INFO (zkConnectionManagerCallback-2967-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1631781 INFO (zkCallback-2927-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1631783 INFO (zkCallback-2927-thread-2) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:53722/solr ready
[junit4] 2> 1631784 INFO (recoveryExecutor-2899-thread-1-processing-n:127.0.0.1:52472_solr x:cdcr-cluster2_shard2_replica_n3 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 1631788 INFO (qtp609129268-11397) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1631788 INFO (qtp609129268-11397) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.c.S.Request [cdcr-cluster2_shard2_replica_n3] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1631789 INFO (qtp800323450-11450) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1604440387261300736,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1631799 INFO (qtp800323450-11450) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 1631800 INFO (qtp800323450-11450) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1631800 INFO (qtp800323450-11450) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2} status=0 QTime=12
[junit4] 2> 1631803 INFO (qtp800323450-11453) [n:127.0.0.1:43249_solr c:cdcr-cluster1 s:shard2 r:core_node4 x:cdcr-cluster1_shard2_replica_n2] o.a.s.c.S.Request [cdcr-cluster1_shard2_replica_n2] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 1631803 INFO (recoveryExecutor-2899-thread-1-processing-n:127.0.0.1:52472_solr x:cdcr-cluster2_shard2_replica_n3 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 1631803 INFO (recoveryExecutor-2899-thread-1-processing-n:127.0.0.1:52472_solr x:cdcr-cluster2_shard2_replica_n3 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 1631803 INFO (recoveryExecutor-2899-thread-1-processing-n:127.0.0.1:52472_solr x:cdcr-cluster2_shard2_replica_n3 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 1631803 INFO (recoveryExecutor-2899-thread-1-processing-n:127.0.0.1:52472_solr x:cdcr-cluster2_shard2_replica_n3 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 1631803 INFO (recoveryExecutor-2899-thread-1-processing-n:127.0.0.1:52472_solr x:cdcr-cluster2_shard2_replica_n3 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 1631808 INFO (recoveryExecutor-2899-thread-1-processing-n:127.0.0.1:52472_solr x:cdcr-cluster2_shard2_replica_n3 c:cdcr-cluster2 s:shard2 r:core_node4) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard2 r:core_node4 x:cdcr-cluster2_shard2_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@70f94ce9[cdcr-cluster2_shard2_replica_n3] main]
[junit4] 2> 1631809 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 1631809 INFO (qtp609129268-11393) [n:127.0.0.1:52472_solr c:cdcr-cluster2 s:shard1 r:core_node2 x:cdcr-cluster2_shard1_replica_n1] o.a.s.c.S.Request [cdcr-cluster2_shard1_replica_n1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=ja
[...truncated too long message...]
ne/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null685139169
[copy] Copying 39 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null685139169
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/null685139169
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1961681413
[copy] Copying 247 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1961681413
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/checkout/solr/null1961681413
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 86ms :: artifacts dl 11ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 375 minutes 54 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)