You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/01/25 04:30:13 UTC
[JENKINS] Lucene-Solr-Tests-7.x - Build # 331 - Still Unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/331/
4 tests failed.
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
max version bucket seed not updated after recovery!
Stack Trace:
java.lang.AssertionError: max version bucket seed not updated after recovery!
at __randomizedtesting.SeedInfo.seed([8A83B0651EC2C427:2D78FBFB03EA9DF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:376)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:131)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestUtilizeNode.test
Error Message:
no replica should be present in 127.0.0.1:45355_solr
Stack Trace:
java.lang.AssertionError: no replica should be present in 127.0.0.1:45355_solr
at __randomizedtesting.SeedInfo.seed([8A83B0651EC2C427:2D78FBFB03EA9DF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.TestUtilizeNode.test(TestUtilizeNode.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeLostTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([8A83B0651EC2C427:8E7D1977240846BD]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeLostTrigger(TriggerIntegrationTest.java:521)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestDetails
Error Message:
Expected to see that the slave has replicated1: {indexSize=117 bytes,indexPath=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-027/./collection1/data/index/,commits=[{indexVersion=1516849324265,generation=2,filelist=[segments_2]}],isMaster=false,isSlave=true,indexVersion=1516849324265,generation=2,slave={masterDetails={indexSize=1.96 KB,indexPath=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-026/./collection1/data/index/,commits=[{indexVersion=0,generation=1,filelist=[segments_1]}, {indexVersion=1516849324265,generation=2,filelist=[_0.cfe, _0.cfs, _0.si, segments_2]}],isMaster=true,isSlave=false,indexVersion=1516849324265,generation=2,master={confFiles=schema.xml,xslt/dummy.xsl,replicateAfter=[commit],replicationEnabled=true,replicableVersion=1516849324265,replicableGeneration=2}},masterUrl=http://127.0.0.1:45157/solr/collection1,pollInterval=00:00:01,nextExecutionAt=Thu Jan 25 08:02:41 TMT 2018,currentDate=Thu Jan 25 08:02:40 TMT 2018,isPollingDisabled=false,isReplicating=false}}
Stack Trace:
java.lang.AssertionError: Expected to see that the slave has replicated1: {indexSize=117 bytes,indexPath=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-027/./collection1/data/index/,commits=[{indexVersion=1516849324265,generation=2,filelist=[segments_2]}],isMaster=false,isSlave=true,indexVersion=1516849324265,generation=2,slave={masterDetails={indexSize=1.96 KB,indexPath=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-026/./collection1/data/index/,commits=[{indexVersion=0,generation=1,filelist=[segments_1]}, {indexVersion=1516849324265,generation=2,filelist=[_0.cfe, _0.cfs, _0.si, segments_2]}],isMaster=true,isSlave=false,indexVersion=1516849324265,generation=2,master={confFiles=schema.xml,xslt/dummy.xsl,replicateAfter=[commit],replicationEnabled=true,replicableVersion=1516849324265,replicableGeneration=2}},masterUrl=http://127.0.0.1:45157/solr/collection1,pollInterval=00:00:01,nextExecutionAt=Thu Jan 25 08:02:41 TMT 2018,currentDate=Thu Jan 25 08:02:40 TMT 2018,isPollingDisabled=false,isReplicating=false}}
at __randomizedtesting.SeedInfo.seed([8A83B0651EC2C427:F0DE9310889AEDAF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.handler.TestReplicationHandler.doTestDetails(TestReplicationHandler.java:332)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12171 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/init-core-data-001
[junit4] 2> 789185 WARN (SUITE-TestReplicationHandler-seed#[8A83B0651EC2C427]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=12 numCloses=12
[junit4] 2> 789185 INFO (SUITE-TestReplicationHandler-seed#[8A83B0651EC2C427]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 789203 INFO (SUITE-TestReplicationHandler-seed#[8A83B0651EC2C427]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 789203 INFO (SUITE-TestReplicationHandler-seed#[8A83B0651EC2C427]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 789204 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTestIndexAndConfigAliasReplication
[junit4] 2> 789204 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-001/collection1
[junit4] 2> 789219 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 789235 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 789235 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 789235 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 789236 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@439d4feb{/solr,null,AVAILABLE}
[junit4] 2> 789237 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3d859021{HTTP/1.1,[http/1.1]}{127.0.0.1:46281}
[junit4] 2> 789237 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.Server Started @799991ms
[junit4] 2> 789237 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=46281}
[junit4] 2> 789237 ERROR (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 789238 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 789238 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 789238 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 789238 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-25T03:00:36.275Z
[junit4] 2> 789238 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-001/solr.xml
[junit4] 2> 789241 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 789241 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 789242 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@38a7c4ad, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 789357 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@38a7c4ad
[junit4] 2> 789395 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@38a7c4ad
[junit4] 2> 789395 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@38a7c4ad
[junit4] 2> 789397 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-001/.
[junit4] 2> 789397 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 789466 INFO (coreLoadExecutor-1395-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 789506 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 789639 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 789700 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 789700 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-001/./collection1, trusted=true
[junit4] 2> 789700 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@38a7c4ad
[junit4] 2> 789700 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 789700 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-001/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-001/./collection1/data/]
[junit4] 2> 789714 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4805299952927429]
[junit4] 2> 789980 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 789980 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 790000 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=96.451171875, floorSegmentMB=1.1083984375, forceMergeDeletesPctAllowed=16.4187459907227, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 790001 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@461aaa9e[collection1] main]
[junit4] 2> 790001 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-001/collection1/conf
[junit4] 2> 790002 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 790002 INFO (coreLoadExecutor-1395-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 790003 INFO (searcherExecutor-1396-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@461aaa9e[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 790017 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-002/collection1
[junit4] 2> 790019 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 790033 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 790033 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 790033 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 790034 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@80453d0{/solr,null,AVAILABLE}
[junit4] 2> 790034 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@389544e7{HTTP/1.1,[http/1.1]}{127.0.0.1:40591}
[junit4] 2> 790034 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.e.j.s.Server Started @800788ms
[junit4] 2> 790034 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=40591}
[junit4] 2> 790034 ERROR (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 790034 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 790034 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 790034 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 790034 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-25T03:00:37.071Z
[junit4] 2> 790035 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-002/solr.xml
[junit4] 2> 790050 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 790050 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 790051 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@38a7c4ad, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 790227 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@38a7c4ad
[junit4] 2> 790277 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@38a7c4ad
[junit4] 2> 790277 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@38a7c4ad
[junit4] 2> 790296 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-002/.
[junit4] 2> 790296 INFO (TEST-TestReplicationHandler.doTestIndexAndConfigAliasReplication-seed#[8A83B0651EC2C427]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 790297 INFO (coreLoadExecutor-1405-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 790365 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 790388 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 790402 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 790402 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-002/./collection1, trusted=true
[junit4] 2> 790402 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@38a7c4ad
[junit4] 2> 790402 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 790402 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-002/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-002/./collection1/data/]
[junit4] 2> 790404 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=15, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4805299952927429]
[junit4] 2> 790621 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 790621 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 790622 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=96.451171875, floorSegmentMB=1.1083984375, forceMergeDeletesPctAllowed=16.4187459907227, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 790622 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4c263013[collection1] main]
[junit4] 2> 790623 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_8A83B0651EC2C427-001/solr-instance-002/collection1/conf
[junit4] 2> 790641 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 790641 INFO (coreLoadExecutor-1405-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 790651 INFO (searcherExecutor-1406-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@4c263013[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 790731 INFO (qtp41072298-3329) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 790792 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 1
[junit4] 2> 790793 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
[junit4] 2> 790793 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
[junit4] 2> 790793 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
[junit4] 2> 790794 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
[junit4] 2> 790794 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
[junit4] 2> 790807 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
[junit4] 2> 790807 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
[junit4] 2> 790807 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
[junit4] 2> 790808 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
[junit4] 2> 790808 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
[junit4] 2> 790809 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
[junit4] 2> 790809 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
[junit4] 2> 790809 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
[junit4] 2> 790810 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
[junit4] 2> 790810 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
[junit4] 2> 790810 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
[junit4] 2> 790827 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
[junit4] 2> 790827 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
[junit4] 2> 790828 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
[junit4] 2> 790829 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
[junit4] 2> 790830 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
[junit4] 2> 790830 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
[junit4] 2> 790831 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
[junit4] 2> 790843 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
[junit4] 2> 790844 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
[junit4] 2> 790844 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
[junit4] 2> 790844 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
[junit4] 2> 790846 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
[junit4] 2> 790846 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
[junit4] 2> 790846 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
[junit4] 2> 790847 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
[junit4] 2> 790897 INFO (qtp41072298-3334) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 790897 INFO (indexFetcher-1410-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 790897 INFO (indexFetcher-1410-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 790897 INFO (indexFetcher-1410-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 790897 INFO (indexFetcher-1410-thread-1) [ x:collection1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 790897 INFO (indexFetcher-1410-thread-1) [ x:collection1] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 790898 INFO (indexFetcher-1410-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 790898 INFO (indexFetcher-1410-thread-1) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@34886db1 commitCommandVersion:0
[junit4] 2> 790899 INFO (indexFetcher-1410-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@33184660[collection1] main]
[junit4] 2> 790899 INFO (indexFetcher-1410-thread-1) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 790900 INFO (searcherExecutor-1406-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@33184660[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 790914 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 66
[junit4] 2> 790914 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
[junit4] 2> 790915 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
[junit4] 2> 790915 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
[junit4] 2> 790915 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
[junit4] 2> 790916 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
[junit4] 2> 790928 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
[junit4] 2> 790929 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
[junit4] 2> 790929 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
[junit4] 2> 790929 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
[junit4] 2> 790930 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
[junit4] 2> 790930 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
[junit4] 2> 790949 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
[junit4] 2> 790949 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
[junit4] 2> 790949 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
[junit4] 2> 790950 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
[junit4] 2> 790950 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
[junit4] 2> 790950 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
[junit4] 2> 790951 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
[junit4] 2> 790951 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
[junit4] 2> 790951 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
[junit4] 2> 790952 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
[junit4] 2> 790952 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
[junit4] 2> 790952 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
[junit4] 2> 790965 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
[junit4] 2> 790965 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
[junit4] 2> 790966 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
[junit4] 2> 790966 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
[junit4] 2> 790966 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
[junit4] 2> 790967 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 0
[junit4] 2> 790967 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
[junit4] 2> 790968 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
[junit4] 2> 790968 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
[junit4] 2> 790968 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
[junit4] 2> 791038 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 12
[junit4] 2> 791038 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 0
[junit4] 2> 791041 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
[junit4] 2> 791041 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
[junit4] 2> 791082 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 40
[junit4] 2> 791083 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
[junit4] 2> 791083 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
[junit4] 2> 791083 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
[junit4] 2> 791084 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
[junit4] 2> 791084 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
[junit4] 2> 791084 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
[junit4] 2> 791085 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
[junit4] 2> 791085 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
[junit4] 2> 791085 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
[junit4] 2> 791086 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
[junit4] 2> 791086 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
[junit4] 2> 791104 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[82]} 0 0
[junit4] 2> 791104 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[83]} 0 0
[junit4] 2> 791105 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[84]} 0 0
[junit4] 2> 791105 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[85]} 0 0
[junit4] 2> 791106 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[86]} 0 0
[junit4] 2> 791119 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[87]} 0 0
[junit4] 2> 791119 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[88]} 0 0
[junit4] 2> 791120 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[89]} 0 0
[junit4] 2> 791120 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[90]} 0 0
[junit4] 2> 791120 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[91]} 0 0
[junit4] 2> 791121 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[92]} 0 0
[junit4] 2> 791121 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[93]} 0 0
[junit4] 2> 791121 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[94]} 0 0
[junit4] 2> 791122 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[95]} 0 0
[junit4] 2> 791122 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[96]} 0 0
[junit4] 2> 791122 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[97]} 0 0
[junit4] 2> 791139 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[98]} 0 0
[junit4] 2> 791139 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[99]} 0 0
[junit4] 2> 791140 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[100]} 0 0
[junit4] 2> 791140 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[101]} 0 0
[junit4] 2> 791140 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[102]} 0 0
[junit4] 2> 791141 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[103]} 0 0
[junit4] 2> 791141 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[104]} 0 0
[junit4] 2> 791141 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[105]} 0 0
[junit4] 2> 791141 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[106]} 0 0
[junit4] 2> 791142 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[107]} 0 0
[junit4] 2> 791142 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[108]} 0 0
[junit4] 2> 791142 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[109]} 0 0
[junit4] 2> 791143 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[110]} 0 0
[junit4] 2> 791155 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[111]} 0 0
[junit4] 2> 791156 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[112]} 0 0
[junit4] 2> 791156 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[113]} 0 0
[junit4] 2> 791157 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[114]} 0 0
[junit4] 2> 791157 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[115]} 0 0
[junit4] 2> 791157 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[116]} 0 0
[junit4] 2> 791158 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[117]} 0 0
[junit4] 2> 791158 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[118]} 0 0
[junit4] 2> 791158 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[119]} 0 0
[junit4] 2> 791159 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[120]} 0 0
[junit4] 2> 791176 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[121]} 0 17
[junit4] 2> 791176 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[122]} 0 0
[junit4] 2> 791177 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[123]} 0 0
[junit4] 2> 791177 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[124]} 0 0
[junit4] 2> 791177 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[125]} 0 0
[junit4] 2> 791178 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[126]} 0 0
[junit4] 2> 791178 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[127]} 0 0
[junit4] 2> 791178 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[128]} 0 0
[junit4] 2> 791179 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[129]} 0 0
[junit4] 2> 791179 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[130]} 0 0
[junit4] 2> 791193 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[131]} 0 0
[junit4] 2> 791194 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[132]} 0 0
[junit4] 2> 791194 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[133]} 0 0
[junit4] 2> 791194 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[134]} 0 0
[junit4] 2> 791195 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[135]} 0 0
[junit4] 2> 791195 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[136]} 0 0
[junit4] 2> 791195 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[137]} 0 0
[junit4] 2> 791212 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[138]} 0 0
[junit4] 2> 791212 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[139]} 0 0
[junit4] 2> 791213 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[140]} 0 0
[junit4] 2> 791213 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[141]} 0 0
[junit4] 2> 791213 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[142]} 0 0
[junit4] 2> 791214 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[143]} 0 0
[junit4] 2> 791214 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[144]} 0 0
[junit4] 2> 791215 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[145]} 0 0
[junit4] 2> 791215 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[146]} 0 0
[junit4] 2> 791228 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[147]} 0 0
[junit4] 2> 791229 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[148]} 0 0
[junit4] 2> 791229 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[149]} 0 0
[junit4] 2> 791229 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[150]} 0 0
[junit4] 2> 791230 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[151]} 0 0
[junit4] 2> 791230 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[152]} 0 0
[junit4] 2> 791230 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[153]} 0 0
[junit4] 2> 791231 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[154]} 0 0
[junit4] 2> 791231 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[155]} 0 0
[junit4] 2> 791231 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[156]} 0 0
[junit4] 2> 791232 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[157]} 0 0
[junit4] 2> 791250 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[158]} 0 0
[junit4] 2> 791250 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[159]} 0 0
[junit4] 2> 791251 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[160]} 0 0
[junit4] 2> 791251 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[161]} 0 0
[junit4] 2> 791251 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[162]} 0 0
[junit4] 2> 791252 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[163]} 0 0
[junit4] 2> 791252 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[164]} 0 0
[junit4] 2> 791265 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[165]} 0 12
[junit4] 2> 791265 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[166]} 0 0
[junit4] 2> 791265 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[167]} 0 0
[junit4] 2> 791266 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[168]} 0 0
[junit4] 2> 791266 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[169]} 0 0
[junit4] 2> 791266 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[170]} 0 0
[junit4] 2> 791266 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[171]} 0 0
[junit4] 2> 791267 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[172]} 0 0
[junit4] 2> 791267 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[173]} 0 0
[junit4] 2> 791267 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[174]} 0 0
[junit4] 2> 791268 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[175]} 0 0
[junit4] 2> 791268 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[176]} 0 0
[junit4] 2> 791268 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[177]} 0 0
[junit4] 2> 791285 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[178]} 0 0
[junit4] 2> 791285 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[179]} 0 0
[junit4] 2> 791286 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[180]} 0 0
[junit4] 2> 791286 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[181]} 0 0
[junit4] 2> 791286 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[182]} 0 0
[junit4] 2> 791287 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[183]} 0 0
[junit4] 2> 791287 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[184]} 0 0
[junit4] 2> 791287 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[185]} 0 0
[junit4] 2> 791288 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[186]} 0 0
[junit4] 2> 791288 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[187]} 0 0
[junit4] 2> 791288 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[188]} 0 0
[junit4] 2> 791301 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[189]} 0 12
[junit4] 2> 791301 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[190]} 0 0
[junit4] 2> 791303 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[191]} 0 0
[junit4] 2> 791303 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[192]} 0 0
[junit4] 2> 791303 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[193]} 0 0
[junit4] 2> 791304 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[194]} 0 0
[junit4] 2> 791304 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[195]} 0 0
[junit4] 2> 791304 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[196]} 0 0
[junit4] 2> 791305 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[197]} 0 0
[junit4] 2> 791305 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[198]} 0 0
[junit4] 2> 791321 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[199]} 0 0
[junit4] 2> 791322 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[200]} 0 0
[junit4] 2> 791322 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[201]} 0 0
[junit4] 2> 791323 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[202]} 0 0
[junit4] 2> 791323 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[203]} 0 0
[junit4] 2> 791323 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[204]} 0 0
[junit4] 2> 791324 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[205]} 0 0
[junit4] 2> 791324 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[206]} 0 0
[junit4] 2> 791324 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[207]} 0 0
[junit4] 2> 791325 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[208]} 0 0
[junit4] 2> 791325 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[209]} 0 0
[junit4] 2> 791338 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[210]} 0 0
[junit4] 2> 791338 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[211]} 0 0
[junit4] 2> 791338 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[212]} 0 0
[junit4] 2> 791339 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[213]} 0 0
[junit4] 2> 791339 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[214]} 0 0
[junit4] 2> 791339 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[215]} 0 0
[junit4] 2> 791339 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[216]} 0 0
[junit4] 2> 791340 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[217]} 0 0
[junit4] 2> 791340 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[218]} 0 0
[junit4] 2> 791340 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[219]} 0 0
[junit4] 2> 791341 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[220]} 0 0
[junit4] 2> 791341 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[221]} 0 0
[junit4] 2> 791341 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[222]} 0 0
[junit4] 2> 791358 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[223]} 0 0
[junit4] 2> 791358 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[224]} 0 0
[junit4] 2> 791359 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[225]} 0 0
[junit4] 2> 791359 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[226]} 0 0
[junit4] 2> 791359 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[227]} 0 0
[junit4] 2> 791360 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[228]} 0 0
[junit4] 2> 791360 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[229]} 0 0
[junit4] 2> 791360 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[230]} 0 0
[junit4] 2> 791361 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[231]} 0 0
[junit4] 2> 791361 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[232]} 0 0
[junit4] 2> 791361 INFO (qtp41072298-3331) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[233]} 0 0
[junit4] 2> 791374 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[234]} 0 12
[junit4] 2> 791374 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[235]} 0 0
[junit4] 2> 791375 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[236]} 0 0
[junit4] 2> 791375 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[237]} 0 0
[junit4] 2> 791375 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[238]} 0 0
[junit4] 2> 791375 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[239]} 0 0
[junit4] 2> 791376 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[240]} 0 0
[junit4] 2> 791376 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[241]} 0 0
[junit4] 2> 791376 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[242]} 0 0
[junit4] 2> 791377 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[243]} 0 0
[junit4] 2> 791377 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[244]} 0 0
[junit4] 2> 791377 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[245]} 0 0
[junit4] 2> 791378 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[246]} 0 0
[junit4] 2> 791394 INFO (qtp41072298-3333) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[247]} 0 0
[junit4] 2> 791394 INFO (qtp41072298-3333) [
[...truncated too long message...]
replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@399175b4: rootName = solr_46641, domain = solr.core.collection1.shard1.replica_n2, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_n2 / com.codahale.metrics.MetricRegistry@1708e4e6
[junit4] 2> 3663078 INFO (coreCloseExecutor-6143-thread-1) [n:127.0.0.1:46641_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1489558529
[junit4] 2> 3663097 INFO (coreCloseExecutor-6139-thread-1) [n:127.0.0.1:44019_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1854206269
[junit4] 2> 3663098 INFO (jetty-closer-4773-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@c97e593{/solr,null,UNAVAILABLE}
[junit4] 2> 3663098 INFO (jetty-closer-4773-thread-5) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3663125 INFO (jetty-closer-4773-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@154c7a95{/solr,null,UNAVAILABLE}
[junit4] 2> 3663125 INFO (jetty-closer-4773-thread-3) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3663165 INFO (zkCallback-4912-thread-2-processing-n:127.0.0.1:44019_solr) [n:127.0.0.1:44019_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44019_solr
[junit4] 2> 3663197 INFO (jetty-closer-4773-thread-7) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@56edf109{/solr,null,UNAVAILABLE}
[junit4] 2> 3663197 INFO (jetty-closer-4773-thread-7) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3663671 WARN (zkCallback-4852-thread-3-processing-n:127.0.0.1:44385_solr) [n:127.0.0.1:44385_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3663671 INFO (jetty-closer-4773-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@b538d62{/solr,null,UNAVAILABLE}
[junit4] 2> 3663672 INFO (jetty-closer-4773-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3663886 WARN (zkCallback-4997-thread-2-processing-n:127.0.0.1:35553_solr) [n:127.0.0.1:35553_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3663922 INFO (jetty-closer-4773-thread-13) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6661039f{/solr,null,UNAVAILABLE}
[junit4] 2> 3663922 INFO (jetty-closer-4773-thread-13) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3663975 ERROR (SUITE-TriggerIntegrationTest-seed#[8A83B0651EC2C427]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3663975 INFO (SUITE-TriggerIntegrationTest-seed#[8A83B0651EC2C427]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37678 37678
[junit4] 2> 3664257 INFO (Thread-4048) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:37678 37678
[junit4] 2> 3664257 WARN (Thread-4048) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 23 /solr/aliases.json
[junit4] 2> 22 /solr/security.json
[junit4] 2> 10 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 89 /solr/collections/collection1/state.json
[junit4] 2> 43 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 23 /solr/clusterprops.json
[junit4] 2> 23 /solr/clusterstate.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 3 /solr/overseer_elect/election/99408405740519478-127.0.0.1:34919_solr-n_0000000016
[junit4] 2> 3 /solr/overseer_elect/election/99408405740519454-127.0.0.1:46641_solr-n_0000000008
[junit4] 2> 3 /solr/overseer_elect/election/99408405740519472-127.0.0.1:40867_solr-n_0000000014
[junit4] 2> 2 /solr/overseer_elect/election/99408405740519487-127.0.0.1:39313_solr-n_0000000019
[junit4] 2> 2 /solr/overseer_elect/election/99408405740519466-127.0.0.1:44019_solr-n_0000000012
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/99408405740519463-core_node8-n_0000000003
[junit4] 2> 2 /solr/overseer_elect/election/99408405740519435-127.0.0.1:35211_solr-n_0000000002
[junit4] 2> 2 /solr/overseer_elect/election/99408405740519460-127.0.0.1:42639_solr-n_0000000010
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 23 /solr/live_nodes
[junit4] 2> 23 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/queue
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_8A83B0651EC2C427-001
[junit4] 2> Jan 25, 2018 3:48:30 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=LuceneFixedGap), x_s=PostingsFormat(name=Memory)}, docValues:{_version_=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1625, maxMBSortInHeap=7.813419516384837, sim=RandomSimilarity(queryNorm=true): {}, locale=hr, timezone=SystemV/HST10
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=163710792,total=501743616
[junit4] 2> NOTE: All tests run in this JVM: [TestQuerySenderNoQuery, CoreAdminRequestStatusTest, TestSmileRequest, TestWordDelimiterFilterFactory, BadIndexSchemaTest, TestCollectionAPIs, TestStressLucene, PKIAuthenticationIntegrationTest, TestDistributedSearch, ConfigSetsAPITest, TestLegacyNumericUtils, TestSolrQueryParser, MissingSegmentRecoveryTest, MetricUtilsTest, TestOnReconnectListenerSupport, TestNamedUpdateProcessors, TestManagedResourceStorage, TestPhraseSuggestions, SubstringBytesRefFilterTest, DistributedFacetSimpleRefinementLongTailTest, SpellingQueryConverterTest, TestReloadDeadlock, TestCloudDeleteByQuery, TestCSVLoader, AutoScalingHandlerTest, TestSubQueryTransformerCrossCore, TestTrieFacet, TestFieldCollectionResource, ChaosMonkeyNothingIsSafeTest, LeaderInitiatedRecoveryOnShardRestartTest, DistributedFacetPivotSmallAdvancedTest, TestRequestStatusCollectionAPI, TestJoin, DistributedFacetExistsSmallTest, UUIDFieldTest, TestSolrDeletionPolicy1, TestMaxTokenLenTokenizer, TestRemoteStreaming, SolrCloudReportersTest, HdfsRecoveryZkTest, TestHashQParserPlugin, TestExecutePlanAction, TestHttpShardHandlerFactory, V2StandaloneTest, BinaryUpdateRequestHandlerTest, TestLeaderInitiatedRecoveryThread, SolrIndexSplitterTest, PreAnalyzedFieldManagedSchemaCloudTest, TestPseudoReturnFields, OverseerTaskQueueTest, TestOmitPositions, BitVectorTest, ShardRoutingTest, BasicDistributedZkTest, TestSlowCompositeReaderWrapper, SimpleFacetsTest, ExternalFileFieldSortTest, TestRawResponseWriter, CdcrReplicationHandlerTest, AtomicUpdateProcessorFactoryTest, AnalyticsQueryTest, ZkNodePropsTest, CdcrUpdateLogTest, SolrMetricsIntegrationTest, LeaderFailoverAfterPartitionTest, TestConfigOverlay, RAMDirectoryFactoryTest, DistributedTermsComponentTest, OpenCloseCoreStressTest, UpdateParamsTest, DistributedQueryElevationComponentTest, TestConfig, DocumentAnalysisRequestHandlerTest, RequestHandlersTest, HdfsDirectoryFactoryTest, TestHdfsCloudBackupRestore, TestWriterPerf, TestUniqueKeyFieldResource, FullSolrCloudDistribCmdsTest, TestComponentsName, TestSQLHandler, HdfsDirectoryTest, OverseerModifyCollectionTest, DeleteInactiveReplicaTest, CollectionsAPIAsyncDistributedZkTest, LargeFieldTest, DistributedQueryComponentCustomSortTest, SolrCoreCheckLockOnStartupTest, TestIndexSearcher, TestReplicationHandlerBackup, ChangedSchemaMergeTest, BlockCacheTest, TestRandomFaceting, TestCopyFieldCollectionResource, JSONWriterTest, JavabinLoaderTest, TestFastOutputStream, SortSpecParsingTest, DebugComponentTest, TestDocumentBuilder, TestFieldSortValues, SpatialRPTFieldTypeTest, TestMergePolicyConfig, CustomCollectionTest, SystemLogListenerTest, IndexBasedSpellCheckerTest, TestPayloadScoreQParserPlugin, TestPointFields, DefaultValueUpdateProcessorTest, AssignBackwardCompatibilityTest, TestFieldCacheSortRandom, OutOfBoxZkACLAndCredentialsProvidersTest, HttpSolrCallGetCoreTest, XmlUpdateRequestHandlerTest, TestMaxScoreQueryParser, TestIBSimilarityFactory, ActionThrottleTest, CloudExitableDirectoryReaderTest, DistribCursorPagingTest, UpdateLogTest, TestRuleBasedAuthorizationPlugin, PluginInfoTest, TestSegmentSorting, ZkStateReaderTest, BJQParserTest, TestCloudSearcherWarming, RecoveryAfterSoftCommitTest, TestClusterProperties, TestConfigSetsAPIExclusivity, TestDeleteCollectionOnDownNodes, TestDistribDocBasedVersion, TestLeaderElectionZkExpiry, TestPullReplicaErrorHandling, TestSkipOverseerOperations, TestTlogReplica, CollectionsAPIDistributedZkTest, NodeLostTriggerTest, TriggerIntegrationTest]
[junit4] Completed [626/767 (4!)] on J0 in 217.03s, 14 tests, 1 failure <<< FAILURES!
[...truncated 48655 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 347 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/347/
16 tests failed.
FAILED: org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataV1
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([72C7600CE6E2864C:A3A7DEC5C64E9515]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.AliasIntegrationTest.checkFooAndBarMeta(AliasIntegrationTest.java:283)
at org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataV1(AliasIntegrationTest.java:251)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataV2
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([72C7600CE6E2864C:FF6A72BEF851AED7]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.AliasIntegrationTest.checkFooAndBarMeta(AliasIntegrationTest.java:283)
at org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataV2(AliasIntegrationTest.java:237)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Could not load collection from ZK: collection1
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: collection1
at __randomizedtesting.SeedInfo.seed([72C7600CE6E2864C:FA935FD6481EEBB4]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
at org.apache.solr.common.cloud.ZkStateReader.forceUpdateCollection(ZkStateReader.java:364)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.updateMappingsFromZk(AbstractFullDistribZkTestBase.java:681)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1318)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1309)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.checkShardConsistency(AbstractFullDistribZkTestBase.java:1301)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexQueryDeleteHierarchical(FullSolrCloudDistribCmdsTest.java:528)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:134)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
... 49 more
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:52968
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:52968
at __randomizedtesting.SeedInfo.seed([72C7600CE6E2864C:FA935FD6481EEBB4]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1674)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1688)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollection(AbstractFullDistribZkTestBase.java:1612)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createCollectionRetry(AbstractFullDistribZkTestBase.java:1917)
at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:354)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:140)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 50 more
FAILED: org.apache.solr.cloud.MoveReplicaHDFSFailoverTest.testOldReplicaIsDeletedInRaceCondition
Error Message:
Could not load collection from ZK: movereplicatest_coll4
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: movereplicatest_coll4
at __randomizedtesting.SeedInfo.seed([72C7600CE6E2864C:7897EF7A26E2E7ED]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1206)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:849)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:173)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:138)
at org.apache.solr.cloud.MoveReplicaHDFSFailoverTest.addDocs(MoveReplicaHDFSFailoverTest.java:203)
at org.apache.solr.cloud.MoveReplicaHDFSFailoverTest.testOldReplicaIsDeletedInRaceCondition(MoveReplicaHDFSFailoverTest.java:173)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/movereplicatest_coll4/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
... 48 more
FAILED: org.apache.solr.cloud.MoveReplicaHDFSFailoverTest.testOldReplicaIsDeleted
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([72C7600CE6E2864C:3D5984D1E7E8C92E]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.MoveReplicaHDFSFailoverTest.testOldReplicaIsDeleted(MoveReplicaHDFSFailoverTest.java:149)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.OverseerTest.testOverseerFailure
Error Message:
core_node1 should be gone after publishing the null state
Stack Trace:
java.lang.AssertionError: core_node1 should be gone after publishing the null state
at __randomizedtesting.SeedInfo.seed([72C7600CE6E2864C:76CFEFFFF447696D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.OverseerTest.testOverseerFailure(OverseerTest.java:646)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.RollingRestartTest.test
Error Message:
KeeperErrorCode = Session expired for /live_nodes
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /live_nodes
at __randomizedtesting.SeedInfo.seed([72C7600CE6E2864C:FA935FD6481EEBB4]:0)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1535)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getChildren$4(SolrZkClient.java:328)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getChildren(SolrZkClient.java:328)
at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:673)
at org.apache.solr.common.cloud.ZkStateReader.updateLiveNodes(ZkStateReader.java:385)
at org.apache.solr.cloud.RollingRestartTest.restartWithRolesTest(RollingRestartTest.java:89)
at org.apache.solr.cloud.RollingRestartTest.test(RollingRestartTest.java:52)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestCloudPseudoReturnFields
Error Message:
Could not load collection from ZK: org.apache.solr.cloud.TestCloudPseudoReturnFields_collection
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: org.apache.solr.cloud.TestCloudPseudoReturnFields_collection
at __randomizedtesting.SeedInfo.seed([72C7600CE6E2864C]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:137)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
at org.apache.solr.cloud.TestCloudPseudoReturnFields.waitForRecoveriesToFinish(TestCloudPseudoReturnFields.java:836)
at org.apache.solr.cloud.TestCloudPseudoReturnFields.createMiniSolrCloudCluster(TestCloudPseudoReturnFields.java:88)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/org.apache.solr.cloud.TestCloudPseudoReturnFields_collection/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
... 29 more
FAILED: org.apache.solr.cloud.TestRandomRequestDistribution.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:37439
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:37439
at __randomizedtesting.SeedInfo.seed([72C7600CE6E2864C:FA935FD6481EEBB4]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:425)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:341)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 44 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestRandomRequestDistribution
Error Message:
39 threads leaked from SUITE scope at org.apache.solr.cloud.TestRandomRequestDistribution: 1) Thread[id=24515, name=ScheduledTrigger-6716-thread-2, state=WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=24485, name=Scheduler-908554138, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=24514, name=qtp1964342669-24514, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=24521, name=searcherExecutor-6724-thread-1, state=WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=24536, name=zkCallback-5561-thread-6, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=24489, name=zkCallback-5561-thread-2, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=24465, name=qtp1964342669-24465, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=24484, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=24464, name=qtp1964342669-24464, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 10) Thread[id=24517, name=searcherExecutor-6719-thread-1, state=WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=24490, name=zkCallback-5561-thread-3, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 12) Thread[id=24510, name=coreZkRegister-6709-thread-1, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=24491, name=zkCallback-5561-thread-4, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 14) Thread[id=24480, name=zkCallback-5561-thread-1, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=24472, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=24537, name=zkCallback-5561-thread-7, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=24473, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 18) Thread[id=24506, name=OverseerStateUpdate-73224306782371851-127.0.0.1:37439_-n_0000000002, state=TIMED_WAITING, group=Overseer state updater.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:415) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:145) at java.lang.Thread.run(Thread.java:748) 19) Thread[id=24519, name=ScheduledTrigger-6716-thread-4, state=WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 20) Thread[id=24505, name=zkConnectionManagerCallback-5562-thread-1-processing-n:127.0.0.1:37439_-EventThread, state=WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 21) Thread[id=24507, name=OverseerCollectionConfigSetProcessor-73224306782371851-127.0.0.1:37439_-n_0000000002, state=TIMED_WAITING, group=Overseer collection creation process.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:392) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:193) at java.lang.Thread.run(Thread.java:748) 22) Thread[id=24508, name=OverseerAutoScalingTriggerThread-73224306782371851-127.0.0.1:37439_-n_0000000002, state=WAITING, group=Overseer autoscaling triggers] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:166) at java.lang.Thread.run(Thread.java:748) 23) Thread[id=24468, name=Scheduler-662695520, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 24) Thread[id=24462, name=qtp1964342669-24462-acceptor-0@a2f6fec-ServerConnector@209f934c{HTTP/1.1,[http/1.1]}{127.0.0.1:37439}, state=RUNNABLE, group=TGRP-TestRandomRequestDistribution] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:379) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:638) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 25) Thread[id=24482, name=Thread-9497, state=WAITING, group=TGRP-TestRandomRequestDistribution] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1722) 26) Thread[id=24518, name=ScheduledTrigger-6716-thread-3, state=WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 27) Thread[id=24461, name=qtp1964342669-24461, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.apache.solr.cloud.OverseerTaskQueue$LatchWatcher.await(OverseerTaskQueue.java:158) at org.apache.solr.cloud.OverseerTaskQueue.offer(OverseerTaskQueue.java:208) at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:314) at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:254) at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:232) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:380) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:530) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.io.ManagedSelector$$Lambda$22/207213238.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 28) Thread[id=24497, name=zkCallback-5561-thread-5, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 29) Thread[id=24488, name=watches-5563-thread-1, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 30) Thread[id=24467, name=qtp1964342669-24467, state=RUNNABLE, group=TGRP-TestRandomRequestDistribution] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 31) Thread[id=24512, name=ScheduledTrigger-6716-thread-1, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at java.lang.Thread.sleep(Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:365) at org.apache.solr.cloud.ZkDistributedQueue.offer(ZkDistributedQueue.java:321) at org.apache.solr.cloud.autoscaling.TriggerEventQueue.offerEvent(TriggerEventQueue.java:58) at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.enqueue(ScheduledTriggers.java:521) at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$add$4(ScheduledTriggers.java:272) at org.apache.solr.cloud.autoscaling.ScheduledTriggers$$Lambda$106/299032129.process(Unknown Source) at org.apache.solr.cloud.autoscaling.NodeLostTrigger.run(NodeLostTrigger.java:159) at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.run(ScheduledTriggers.java:569) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 32) Thread[id=24527, name=updateExecutor-5557-thread-1, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 33) Thread[id=24463, name=qtp1964342669-24463, state=RUNNABLE, group=TGRP-TestRandomRequestDistribution] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 34) Thread[id=24504, name=zkConnectionManagerCallback-5562-thread-1-processing-n:127.0.0.1:37439_-SendThread(127.0.0.1:51810), state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054) 35) Thread[id=24509, name=OverseerThreadFactory-6718-thread-1, state=WAITING, group=Overseer collection creation process.] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 36) Thread[id=24479, name=zkConnectionManagerCallback-5562-thread-1, state=WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 37) Thread[id=24528, name=recoveryExecutor-5558-thread-1, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 38) Thread[id=24460, name=qtp1964342669-24460, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 39) Thread[id=24466, name=qtp1964342669-24466, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 39 threads leaked from SUITE scope at org.apache.solr.cloud.TestRandomRequestDistribution:
1) Thread[id=24515, name=ScheduledTrigger-6716-thread-2, state=WAITING, group=TGRP-TestRandomRequestDistribution]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=24485, name=Scheduler-908554138, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=24514, name=qtp1964342669-24514, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.
[...truncated too long message...]
n(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 20) Thread[id=24467, name=qtp1964342669-24467, state=RUNNABLE, group=TGRP-TestRandomRequestDistribution]
[junit4] > at sun.nio.ch.IOUtil.drain(Native Method)
[junit4] > at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304)
[junit4] > at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179)
[junit4] > at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] > at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 21) Thread[id=24512, name=ScheduledTrigger-6716-thread-1, state=WAITING, group=TGRP-TestRandomRequestDistribution]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
[junit4] > at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 22) Thread[id=24527, name=updateExecutor-5557-thread-1, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 23) Thread[id=24463, name=qtp1964342669-24463, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 24) Thread[id=24504, name=zkConnectionManagerCallback-5562-thread-1-processing-n:127.0.0.1:37439_-SendThread(127.0.0.1:51810), state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution]
[junit4] > at java.lang.Thread.sleep(Native Method)
[junit4] > at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000)
[junit4] > at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063)
[junit4] > 25) Thread[id=24479, name=zkConnectionManagerCallback-5562-thread-1, state=WAITING, group=TGRP-TestRandomRequestDistribution]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] > at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 26) Thread[id=24528, name=recoveryExecutor-5558-thread-1, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
[junit4] > at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
[junit4] > at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 27) Thread[id=24460, name=qtp1964342669-24460, state=RUNNABLE, group=TGRP-TestRandomRequestDistribution]
[junit4] > at sun.nio.ch.IOUtil.drain(Native Method)
[junit4] > at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104)
[junit4] > at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] > at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375)
[junit4] > at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304)
[junit4] > at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179)
[junit4] > at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] > at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > 28) Thread[id=24466, name=qtp1964342669-24466, state=TIMED_WAITING, group=TGRP-TestRandomRequestDistribution]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([72C7600CE6E2864C]:0)
[junit4] Completed [549/774 (12!)] on J0 in 100.37s, 1 test, 3 errors <<< FAILURES!
[...truncated 48886 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 346 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/346/
11 tests failed.
FAILED: org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal
Error Message:
concurrent alias creation failed java.lang.IllegalStateException: Connection pool shut down
Stack Trace:
java.lang.AssertionError: concurrent alias creation failed java.lang.IllegalStateException: Connection pool shut down
at __randomizedtesting.SeedInfo.seed([5E759413D09EF7AE:9AEE98417D814094]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal(ConcurrentCreateRoutedAliasTest.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader
Error Message:
Didn't see all replicas for shard shard1 in forceleader_test_collection come up within 90000 ms! ClusterState: { "control_collection":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{"core_node2":{ "core":"control_collection_shard1_replica_n1", "base_url":"http://127.0.0.1:50147", "node_name":"127.0.0.1:50147_", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}, "forceleader_test_collection":{ "pullReplicas":"0", "replicationFactor":"0", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node2":{ "core":"forceleader_test_collection_shard1_replica_t1", "base_url":"http://127.0.0.1:50740", "node_name":"127.0.0.1:50740_", "state":"active", "type":"TLOG", "leader":"true"}, "core_node4":{ "core":"forceleader_test_collection_shard1_replica_t3", "base_url":"http://127.0.0.1:36186", "node_name":"127.0.0.1:36186_", "state":"down", "type":"TLOG"}, "core_node6":{ "core":"forceleader_test_collection_shard1_replica_t5", "base_url":"http://127.0.0.1:51355", "node_name":"127.0.0.1:51355_", "state":"down", "type":"TLOG"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"0", "tlogReplicas":"3"}, "collection1":{ "pullReplicas":"0", "replicationFactor":"1", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{"core_node44":{ "core":"collection1_shard1_replica_t43", "base_url":"http://127.0.0.1:36186", "node_name":"127.0.0.1:36186_", "state":"active", "type":"TLOG", "leader":"true"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node42":{ "core":"collection1_shard2_replica_t41", "base_url":"http://127.0.0.1:50740", "node_name":"127.0.0.1:50740_", "state":"active", "type":"TLOG", "leader":"true"}, "core_node46":{ "core":"collection1_shard2_replica_t45", "base_url":"http://127.0.0.1:51355", "node_name":"127.0.0.1:51355_", "state":"down", "type":"TLOG"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"1", "tlogReplicas":"0"}}
Stack Trace:
java.lang.AssertionError: Didn't see all replicas for shard shard1 in forceleader_test_collection come up within 90000 ms! ClusterState: {
"control_collection":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{"core_node2":{
"core":"control_collection_shard1_replica_n1",
"base_url":"http://127.0.0.1:50147",
"node_name":"127.0.0.1:50147_",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"},
"forceleader_test_collection":{
"pullReplicas":"0",
"replicationFactor":"0",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node2":{
"core":"forceleader_test_collection_shard1_replica_t1",
"base_url":"http://127.0.0.1:50740",
"node_name":"127.0.0.1:50740_",
"state":"active",
"type":"TLOG",
"leader":"true"},
"core_node4":{
"core":"forceleader_test_collection_shard1_replica_t3",
"base_url":"http://127.0.0.1:36186",
"node_name":"127.0.0.1:36186_",
"state":"down",
"type":"TLOG"},
"core_node6":{
"core":"forceleader_test_collection_shard1_replica_t5",
"base_url":"http://127.0.0.1:51355",
"node_name":"127.0.0.1:51355_",
"state":"down",
"type":"TLOG"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"0",
"tlogReplicas":"3"},
"collection1":{
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{"core_node44":{
"core":"collection1_shard1_replica_t43",
"base_url":"http://127.0.0.1:36186",
"node_name":"127.0.0.1:36186_",
"state":"active",
"type":"TLOG",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node42":{
"core":"collection1_shard2_replica_t41",
"base_url":"http://127.0.0.1:50740",
"node_name":"127.0.0.1:50740_",
"state":"active",
"type":"TLOG",
"leader":"true"},
"core_node46":{
"core":"collection1_shard2_replica_t45",
"base_url":"http://127.0.0.1:51355",
"node_name":"127.0.0.1:51355_",
"state":"down",
"type":"TLOG"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"1",
"tlogReplicas":"0"}}
at __randomizedtesting.SeedInfo.seed([5E759413D09EF7AE:B8E2A0D3E91C0ECF]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.ensureAllReplicasAreActive(AbstractFullDistribZkTestBase.java:2005)
at org.apache.solr.cloud.ForceLeaderTest.testReplicasInLIRNoLeader(ForceLeaderTest.java:237)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestPullReplica.testCreateDelete {seed=[5E759413D09EF7AE:4565655F266E0F9E]}
Error Message:
KeeperErrorCode = Session expired for /
Stack Trace:
org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.SolrZkClient.printLayout(SolrZkClient.java:569)
at org.apache.solr.common.cloud.SolrZkClient.printLayoutToStream(SolrZkClient.java:614)
at org.apache.solr.common.cloud.SolrZkClient.printLayoutToStdOut(SolrZkClient.java:609)
at org.apache.solr.cloud.TestPullReplica.testCreateDelete(TestPullReplica.java:189)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestPullReplica.testCreateDelete {seed=[5E759413D09EF7AE:F133D9A312ACC4B2]}
Error Message:
Expected exception SolrException but no exception was thrown
Stack Trace:
junit.framework.AssertionFailedError: Expected exception SolrException but no exception was thrown
at __randomizedtesting.SeedInfo.seed([5E759413D09EF7AE:F133D9A312ACC4B2]:0)
at org.apache.lucene.util.LuceneTestCase.expectThrows(LuceneTestCase.java:2675)
at org.apache.solr.cloud.TestPullReplica.setUp(TestPullReplica.java:102)
at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:968)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitAfterFailedSplit
Error Message:
expected:<1> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<2>
at __randomizedtesting.SeedInfo.seed([5E759413D09EF7AE:A73807BCECEBBA24]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitAfterFailedSplit(ShardSplitTest.java:282)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test
Error Message:
Could not load collection from ZK: hdfsbackuprestore_restored
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: hdfsbackuprestore_restored
at __randomizedtesting.SeedInfo.seed([5E759413D09EF7AE:D621ABC97E629A56]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:137)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:291)
at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:142)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/hdfsbackuprestore_restored/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
... 44 more
FAILED: org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest.testReadApi
Error Message:
expected:<1> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<1> but was:<0>
at __randomizedtesting.SeedInfo.seed([5E759413D09EF7AE:95C6FA60B6C15B5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest.testReadApi(AutoScalingHandlerTest.java:725)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeMarkersRegistration
Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:49559_solr wasn't created
Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:49559_solr wasn't created
at __randomizedtesting.SeedInfo.seed([5E759413D09EF7AE:46CF1C1FDEAB3A41]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeMarkersRegistration(TriggerIntegrationTest.java:938)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([5E759413D09EF7AE:33D8A9A1F5851E1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testCooldown
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([5E759413D09EF7AE:6FCBF9F7AE34825C]:0)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testCooldown(TestTriggerIntegration.java:1086)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.test
Error Message:
Could not load collection from ZK: myalias_2017-10-23
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: myalias_2017-10-23
at __randomizedtesting.SeedInfo.seed([5E759413D09EF7AE:D621ABC97E629A56]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
at org.apache.solr.client.solrj.impl.CloudSolrClient.getDocCollection(CloudSolrClient.java:1206)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:849)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.queryNumDocs(TimeRoutedAliasUpdateProcessorTest.java:286)
at org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.addDocsAndCommit(TimeRoutedAliasUpdateProcessorTest.java:232)
at org.apache.solr.update.processor.TimeRoutedAliasUpdateProcessorTest.test(TimeRoutedAliasUpdateProcessorTest.java:163)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/myalias_2017-10-23/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
... 48 more
Build Log:
[...truncated 11774 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.AutoScalingHandlerTest
[junit4] 2> 0 INFO (SUITE-AutoScalingHandlerTest-seed#[5E759413D09EF7AE]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.AutoScalingHandlerTest_5E759413D09EF7AE-001/init-core-data-001
[junit4] 2> 1747 INFO (SUITE-AutoScalingHandlerTest-seed#[5E759413D09EF7AE]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2425 INFO (SUITE-AutoScalingHandlerTest-seed#[5E759413D09EF7AE]-worker) [ ] o.e.j.u.log Logging initialized @25983ms to org.eclipse.jetty.util.log.Slf4jLog
[junit4] 2> 2509 INFO (SUITE-AutoScalingHandlerTest-seed#[5E759413D09EF7AE]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 5902 INFO (SUITE-AutoScalingHandlerTest-seed#[5E759413D09EF7AE]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.AutoScalingHandlerTest_5E759413D09EF7AE-001/tempDir-001
[junit4] 2> 5974 INFO (SUITE-AutoScalingHandlerTest-seed#[5E759413D09EF7AE]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 6009 INFO (Thread-1) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 6010 INFO (Thread-1) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 6760 INFO (SUITE-AutoScalingHandlerTest-seed#[5E759413D09EF7AE]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:41506
[junit4] 2> 7708 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Exception causing close of session 0x0: ZooKeeperServer not running
[junit4] 2> 8503 ERROR (Thread-1) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 10257 INFO (zkConnectionManagerCallback-4-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 10532 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1042160735b0000, likely client has closed socket
[junit4] 2> 14084 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 14119 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 14361 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 14361 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 14373 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 15381 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 15437 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 15438 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 15789 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4d23014d{/solr,null,AVAILABLE}
[junit4] 2> 15790 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1830188c{/solr,null,AVAILABLE}
[junit4] 2> 16641 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@22dc005a{SSL,[ssl, http/1.1]}{127.0.0.1:38245}
[junit4] 2> 16642 INFO (jetty-launcher-1-thread-1) [ ] o.e.j.s.Server Started @40325ms
[junit4] 2> 16642 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38245}
[junit4] 2> 16810 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2c764106{SSL,[ssl, http/1.1]}{127.0.0.1:58364}
[junit4] 2> 16810 INFO (jetty-launcher-1-thread-2) [ ] o.e.j.s.Server Started @40493ms
[junit4] 2> 16810 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58364}
[junit4] 2> 16911 ERROR (jetty-launcher-1-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 16928 ERROR (jetty-launcher-1-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 16929 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 16948 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 16982 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 16982 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 16982 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 16984 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 17724 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-01T12:25:35.886Z
[junit4] 2> 17761 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-01T12:25:35.885Z
[junit4] 2> 18201 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 18336 INFO (zkConnectionManagerCallback-8-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 18377 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 18398 INFO (zkConnectionManagerCallback-6-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 18657 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 26012 INFO (jetty-launcher-1-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41506/solr
[junit4] 2> 26037 INFO (jetty-launcher-1-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41506/solr
[junit4] 2> 26286 INFO (zkConnectionManagerCallback-14-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 26309 INFO (zkConnectionManagerCallback-16-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 26805 INFO (zkConnectionManagerCallback-20-thread-1-processing-n:127.0.0.1:58364_solr) [n:127.0.0.1:58364_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 26806 INFO (zkConnectionManagerCallback-18-thread-1-processing-n:127.0.0.1:38245_solr) [n:127.0.0.1:38245_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 32613 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:38245_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 32635 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:38245_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38245_solr
[junit4] 2> 33009 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:38245_solr ] o.a.s.c.Overseer Overseer (id=73220192080691205-127.0.0.1:38245_solr-n_0000000000) starting
[junit4] 2> 33203 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:58364_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 33243 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:58364_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58364_solr
[junit4] 2> 33555 INFO (zkCallback-19-thread-1-processing-n:127.0.0.1:58364_solr) [n:127.0.0.1:58364_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 33746 INFO (zkCallback-17-thread-1-processing-n:127.0.0.1:38245_solr) [n:127.0.0.1:38245_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 35184 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:38245_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38245_solr
[junit4] 2> 35363 INFO (zkCallback-19-thread-1-processing-n:127.0.0.1:58364_solr) [n:127.0.0.1:58364_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 35381 INFO (OverseerStateUpdate-73220192080691205-127.0.0.1:38245_solr-n_0000000000) [n:127.0.0.1:38245_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 40910 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:58364_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58364.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29bb994b
[junit4] 2> 40911 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:58364_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58364.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29bb994b
[junit4] 2> 40911 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:58364_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58364.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29bb994b
[junit4] 2> 40929 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:38245_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38245.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29bb994b
[junit4] 2> 40970 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:38245_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38245.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29bb994b
[junit4] 2> 40970 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:38245_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38245.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@29bb994b
[junit4] 2> 41672 INFO (jetty-launcher-1-thread-2) [n:127.0.0.1:58364_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.AutoScalingHandlerTest_5E759413D09EF7AE-001/tempDir-001/node2/.
[junit4] 2> 41678 INFO (jetty-launcher-1-thread-1) [n:127.0.0.1:38245_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.AutoScalingHandlerTest_5E759413D09EF7AE-001/tempDir-001/node1/.
[junit4] 2> 43940 INFO (zkConnectionManagerCallback-26-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 44167 INFO (zkConnectionManagerCallback-30-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 44206 INFO (SUITE-AutoScalingHandlerTest-seed#[5E759413D09EF7AE]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 44208 INFO (SUITE-AutoScalingHandlerTest-seed#[5E759413D09EF7AE]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41506/solr ready
[junit4] 2> 44645 INFO (TEST-AutoScalingHandlerTest.testReadApi-seed#[5E759413D09EF7AE]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testReadApi
[junit4] 2> 51915 INFO (qtp1363753776-24) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=41
[junit4] 2> 54345 INFO (qtp1859038655-37) [n:127.0.0.1:58364_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=17
[junit4] 2> 54389 INFO (qtp1363753776-27) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=4184
[junit4] 2> 54618 INFO (qtp1363753776-29) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 54684 INFO (qtp1859038655-36) [n:127.0.0.1:58364_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 54744 INFO (qtp1859038655-37) [n:127.0.0.1:58364_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=334
[junit4] 2> 54864 INFO (qtp1363753776-30) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName,os.systemLoadAverage,memory.heap.usage&wt=javabin&version=2&group=solr.jvm,solr.node,solr.core} status=0 QTime=3
[junit4] 2> 54870 INFO (qtp1859038655-39) [n:127.0.0.1:58364_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName,os.systemLoadAverage,memory.heap.usage&wt=javabin&version=2&group=solr.jvm,solr.node,solr.core} status=0 QTime=1
[junit4] 2> 54874 INFO (qtp1363753776-27) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=119
[junit4] 2> 54921 INFO (qtp1363753776-24) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName,os.systemLoadAverage,memory.heap.usage&wt=javabin&version=2&group=solr.jvm,solr.node,solr.core} status=0 QTime=14
[junit4] 2> 54997 INFO (qtp1859038655-37) [n:127.0.0.1:58364_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName,os.systemLoadAverage,memory.heap.usage&wt=javabin&version=2&group=solr.jvm,solr.node,solr.core} status=0 QTime=0
[junit4] 2> 55024 INFO (qtp1363753776-27) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=145
[junit4] 2> 55094 INFO (qtp1363753776-27) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=48
[junit4] 2> 55170 INFO (qtp1363753776-30) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName,os.systemLoadAverage,memory.heap.usage&wt=javabin&version=2&group=solr.jvm,solr.node,solr.core} status=0 QTime=0
[junit4] 2> 55190 INFO (qtp1859038655-39) [n:127.0.0.1:58364_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName,os.systemLoadAverage,memory.heap.usage&wt=javabin&version=2&group=solr.jvm,solr.node,solr.core} status=0 QTime=0
[junit4] 2> 55230 INFO (qtp1363753776-27) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling/diagnostics params={wt=javabin&version=2} status=0 QTime=104
[junit4] 2> 55433 INFO (qtp1363753776-27) [n:127.0.0.1:38245_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=6&collection.configName=conf&maxShardsPerNode=10&name=readApiTestViolations&nrtReplicas=6&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 55461 INFO (OverseerThreadFactory-12-thread-1-processing-n:127.0.0.1:38245_solr) [n:127.0.0.1:38245_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection readApiTestViolations
[junit4] 2> 55509 ERROR (OverseerThreadFactory-12-thread-1-processing-n:127.0.0.1:38245_solr) [n:127.0.0.1:38245_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection: readApiTestViolations operation: create failed:org.apache.solr.common.SolrException: 'maxShardsPerNode>0' is not supported when autoScaling policies are used
[junit4] 2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.buildReplicaPositions(CreateCollectionCmd.java:293)
[junit4] 2> at org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:122)
[junit4] 2> at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:254)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:469)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 56199 INFO (qtp1363753776-27) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=6&collection.configName=conf&maxShardsPerNode=10&name=readApiTestViolations&nrtReplicas=6&action=CREATE&numShards=1&wt=javabin&version=2} status=400 QTime=934
[junit4] 2> 56231 INFO (OverseerCollectionConfigSetProcessor-73220192080691205-127.0.0.1:38245_solr-n_0000000000) [n:127.0.0.1:38245_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 56282 INFO (qtp1363753776-24) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName,os.systemLoadAverage,memory.heap.usage&wt=javabin&version=2&group=solr.jvm,solr.node,solr.core} status=0 QTime=0
[junit4] 2> 56407 INFO (qtp1859038655-37) [n:127.0.0.1:58364_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName,os.systemLoadAverage,memory.heap.usage&wt=javabin&version=2&group=solr.jvm,solr.node,solr.core} status=0 QTime=0
[junit4] 2> 56437 INFO (qtp1363753776-27) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=175
[junit4] 2> 56463 INFO (qtp1363753776-29) [n:127.0.0.1:38245_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=6&collection.configName=conf&name=readApiTestViolations&nrtReplicas=6&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 56581 INFO (OverseerThreadFactory-12-thread-2-processing-n:127.0.0.1:38245_solr) [n:127.0.0.1:38245_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection readApiTestViolations
[junit4] 2> 56597 WARN (OverseerThreadFactory-12-thread-2-processing-n:127.0.0.1:38245_solr) [n:127.0.0.1:38245_solr ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 6 on collection readApiTestViolations is higher than the number of Solr instances currently live or live and part of your createNodeSet(2). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 56793 INFO (qtp1363753776-30) [n:127.0.0.1:38245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName,os.systemLoadAverage,memory.heap.usage&wt=javabin&version=2&group=solr.jvm,solr.node,solr.core} status=0 QTime=18
[junit4] 2> 56839 INFO (qtp1859038655-39) [n:127.0.0.1:58364_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName,os.systemLoadAverage,memory.heap.usage&wt=javabin&version=2&group=solr.jvm,solr.node,solr.core} status=0 QTime=0
[junit4] 2> 57345 INFO (OverseerThreadFactory-12-thread-2-processing-n:127.0.0.1:38245_solr) [n:127.0.0.1:38245_solr ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 17739888860 sessionWrapper.createTime 17739888356414277, this.sessionWrapper.createTime 17739888356414277
[junit4] 2> 57602 INFO (OverseerStateUpdate-73220192080691205-127.0.0.1:38245_solr-n_0000000000) [n:127.0.0.1:38245_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"readApiTestViolations",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"readApiTestViolations_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38245/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 57935 INFO (OverseerStateUpdate-73220192080691205-127.0.0.1:38245_solr-n_0000000000) [n:127.0.0.1:38245_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"readApiTestViolations",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"readApiTestViolations_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:58364/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 57940 INFO (OverseerStateUpdate-73220192080691205-127.0.0.1:38245_solr-n_0000000000) [n:127.0.0.1:38245_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"readApiTestViolations",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"readApiTestViolations_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38245/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 57951 INFO (OverseerStateUpdate-73220192080691205-127.0.0.1:38245_solr-n_0000000000) [n:127.0.0.1:38245_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"readApiTestViolations",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"readApiTestViolations_shard1_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:58364/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 57972 INFO (OverseerStateUpdate-73220192080691205-127.0.0.1:38245_solr-n_0000000000) [n:127.0.0.1:38245_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"readApiTestViolations",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"readApiTestViolations_shard1_replica_n9",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38245/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 57992 INFO (OverseerStateUpdate-73220192080691205-127.0.0.1:38245_solr-n_0000000000) [n:127.0.0.1:38245_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"readApiTestViolations",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"readApiTestViolations_shard1_replica_n10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:58364/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 58567 INFO (qtp1363753776-24) [n:127.0.0.1:38245_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=readApiTestViolations_shard1_replica_n1&action=CREATE&numShards=1&collection=readApiTestViolations&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 58659 INFO (qtp1363753776-24) [n:127.0.0.1:38245_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 58645 INFO (qtp1363753776-97) [n:127.0.0.1:38245_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf&newCollection=true&name=readApiTestViolations_shard1_replica_n9&action=CREATE&numShards=1&collection=readApiTestViolations&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 58645 INFO (qtp1859038655-33) [n:127.0.0.1:58364_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=readApiTestViolations_shard1_replica_n3&action=CREATE&numShards=1&collection=readApiTestViolations&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 58958 INFO (qtp1859038655-33) [n:127.0.0.1:58364_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 59182 INFO (qtp1363753776-27) [n:127.0.0.1:38245_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=readApiTestViolations_shard1_replica_n4&action=CREATE&numShards=1&collection=readApiTestViolations&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 59183 INFO (qtp1859038655-39) [n:127.0.0.1:58364_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node12&collection.configName=conf&newCollection=true&name=readApiTestViolations_shard1_replica_n10&action=CREATE&numShards=1&collection=readApiTestViolations&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 59205 INFO (qtp1859038655-36) [n:127.0.0.1:58364_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=readApiTestViolations_shard1_replica_n6&action=CREATE&numShards=1&collection=readApiTestViolations&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 59376 INFO (zkCallback-17-thread-1-processing-n:127.0.0.1:38245_solr) [n:127.0.0.1:38245_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/readApiTestViolations/state.json] for collection [readApiTestViolations] has occurred - updating... (live nodes size: [2])
[junit4] 2> 59376 INFO (zkCallback-19-thread-1-processing-n:127.0.0.1:58364_solr) [n:127.0.0.1:58364_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/readApiTestViolations/state.json] for collection [readApiTestViolations] has occurred - updating... (live nodes size: [2])
[junit4] 2> 59434 INFO (zkCallback-19-thread-2-processing-n:127.0.0.1:58364_solr) [n:127.0.0.1:58364_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/readApiTestViolations/state.json] for collection [readApiTestViolations] has occurred - updating... (live nodes size: [2])
[junit4] 2> 60573 INFO (qtp1363753776-24) [n:127.0.0.1:38245_solr c:readApiTestViolations s:shard1 r:core_node2 x:readApiTestViolations_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 60688 INFO (qtp1859038655-36) [n:127.0.0.1:58364_solr c:readApiTestViolations s:shard1 r:core_node8 x:readApiTestViolations_shard1_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 60692 INFO (qtp1859038655-33) [n:127.0.0.1:58364_solr c:readApiTestViolations s:shard1 r:core_node5 x:readApiTestViolations_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 60593 INFO (qtp1363753776-27) [n:127.0.0.1:38245_solr c:readApiTestViolations s:shard1 r:core_node7 x:readApiTestViolations_shard1_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 60696 INFO (qtp1363753776-97) [n:127.0.0.1:38245_solr c:readApiTestViolations s:shard1 r:core_node11 x:readApiTestViolations_shard1_replica_n9] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 60759 INFO (qtp1859038655-39) [n:127.0.0.1:58364_solr c:readApiTestViolations s:shard1 r:core_node12 x:readApiTestViolations_shard1_replica_n10] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 63337 INFO (qtp1363753776-97) [n:127.0.0.1:38245_solr c:readApiTestViolations s:shard1 r:core_node11 x:readApiTestViolations_shard1_replica_n9] o.a.s.s.IndexSchema [readApiTestViolations_shard1_replica_n9] Schema name=minimal
[junit4] 2> 63339 INFO (qtp1859038655-39) [n:127.0.0.1:58364_solr c:readApiTestViolations s:shard1 r:core_node12 x:readApiTestViolations_shard1_replica_n10] o.a.s.s.IndexSchema [readApiTestViolations_shard1_replica_n10] Schema name=minimal
[junit4] 2> 63510 INFO (qtp1363753776-27) [n:127.0.0.1:38245_solr c:readApiTestViolations s:shard1 r:core_node7 x:readApiTestViolations_shard1_replica_n4] o.a.s.s.IndexSchema [readApiTestViolations_shard1_replica_n4] Schema name=minimal
[junit4] 2> 63522 INFO (qtp1363753776-24) [n:127.0.0.1:38245_solr c:readApiTestViolations s:shard1 r:core_node2 x:readApiTestViolations_shard1_replica_n1] o.a.s.s.IndexSchema [readApiTestViolations_shard1_replica_n1] Schema name=minimal
[junit4] 2> 64260 WARN (qtp1363753776-97) [n:127.0.0.1:38245_solr c:readApiTestViolations s:shard1 r:core_node11 x:readApiTestViolations_shard1_replica_n9] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieIntField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 64101 WARN (qtp1363753776-27) [n:127.0.0.1:38245_solr c:readApiTestViolations s:shard1 r:core_node7 x:readApiTestViolations_shard1_replica_n4] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieIntField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 64100 WARN (qtp1859038655-39) [n:127.0.0.1:58364_solr c:readApiTestViolations s:shard1 r:core_node12 x:readApiTestViolations_shard1_replica_n10] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieIntField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 63956 INFO (qtp1859038655-33) [n:127.0.0.1:58364_solr c:readApiTestViolations s:shard1 r:core_node5 x:readApiTestViolations_shard1_replica_n3] o.a.s.s.IndexSchema [readApiTestViolations_shard1_replica_n3] Schema name=minimal
[junit4] 2> 63924 INFO (qtp1859038655-36) [n:127.0.0.1:58364_solr c:readApiTestViolations s:shard1 r:core_node8 x:readApiTestViolations_shard1_replica_n6] o.a.s.s.IndexSchema [readApiTestViolations_shard1_replica_n6] Schema name=minimal
[junit4] 2> 64605 WARN (qtp1363753776-27) [n:127.0.0.1:38245_solr c:readApiTestViolations s:shard1 r:core_node7 x:readApiTestViolations_shard1_replica_n4] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieLongField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 64624 WARN (qtp1859038655-36) [n:127.0.0.1:58364_solr c:readApiTestViolations s:shard1 r:core_node8 x:readApiTestViolations_shard1_replica_n6] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.TrieLongField]. Pleas
[...truncated too long message...]
2> 10928548 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[5E759413D09EF7AE]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 10928548 INFO (zkCallback-6901-thread-1-processing-n:127.0.0.1:47248_ld_%2Fax) [n:127.0.0.1:47248_ld_%2Fax ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 10928566 INFO (coreCloseExecutor-7135-thread-1) [n:127.0.0.1:47248_ld_%2Fax c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.c.SolrCore [collection1_shard1_replica_t47] CLOSING SolrCore org.apache.solr.core.SolrCore@66bd7fe9
[junit4] 2> 10928566 INFO (coreCloseExecutor-7135-thread-1) [n:127.0.0.1:47248_ld_%2Fax c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_t47, tag=1723695081
[junit4] 2> 10928566 INFO (coreCloseExecutor-7135-thread-1) [n:127.0.0.1:47248_ld_%2Fax c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@780f3ed9: rootName = null, domain = solr.core.collection1.shard1.replica_t47, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_t47 / com.codahale.metrics.MetricRegistry@173c496b
[junit4] 2> 10928732 INFO (coreCloseExecutor-7135-thread-1) [n:127.0.0.1:47248_ld_%2Fax c:collection1 s:shard1 r:core_node48 x:collection1_shard1_replica_t47] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1723695081
[junit4] 2> 10928737 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[5E759413D09EF7AE]) [ ] o.a.s.c.Overseer Overseer (id=73220899329015827-127.0.0.1:47248_ld_%2Fax-n_0000000004) closing
[junit4] 2> 10928738 INFO (OverseerStateUpdate-73220899329015827-127.0.0.1:47248_ld_%2Fax-n_0000000004) [n:127.0.0.1:47248_ld_%2Fax ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:47248_ld_%2Fax
[junit4] 2> 10928738 WARN (OverseerAutoScalingTriggerThread-73220899329015827-127.0.0.1:47248_ld_%2Fax-n_0000000004) [n:127.0.0.1:47248_ld_%2Fax ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 10928794 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[5E759413D09EF7AE]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@53aa91e3{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 10928794 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[5E759413D09EF7AE]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5b60d898{/ld_/ax,null,UNAVAILABLE}
[junit4] 2> 10928794 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[5E759413D09EF7AE]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 10929103 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[5E759413D09EF7AE]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 33819
[junit4] 2> 10929103 ERROR (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[5E759413D09EF7AE]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 10929130 INFO (TEST-ShardSplitTest.testSplitStaticIndexReplication-seed#[5E759413D09EF7AE]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:55377 55377
[junit4] 2> 10929422 INFO (Thread-4420) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:55377 55377
[junit4] 2> 10929422 WARN (Thread-4420) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/aliases.json
[junit4] 2> 6 /solr/security.json
[junit4] 2> 6 /solr/configs/conf1
[junit4] 2> 6 /solr/collections/collection1/terms/shard1
[junit4] 2> 6 /solr/collections/testSplitStaticIndexReplication/terms/shard1_0
[junit4] 2> 5 /solr/collections/collection1/state.json
[junit4] 2> 5 /solr/collections/collection1/terms/shard2
[junit4] 2> 3 /solr/collections/testSplitStaticIndexReplication/terms/shard1_1
[junit4] 2> 3 /solr/collections/testSplitStaticIndexReplication/terms/shard1
[junit4] 2> 2 /solr/collections/control_collection/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/clusterprops.json
[junit4] 2> 10 /solr/clusterstate.json
[junit4] 2> 4 /solr/collections/testSplitStaticIndexReplication/state.json
[junit4] 2> 4 /solr/autoscaling.json
[junit4] 2> 2 /solr/overseer_elect/election/73220899329015817-127.0.0.1:46629_ld_%2Fax-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/live_nodes
[junit4] 2> 10 /solr/collections
[junit4] 2> 4 /solr/overseer/queue
[junit4] 2> 4 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 4 /solr/overseer/collection-queue-work
[junit4] 2> 4 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.ShardSplitTest_5E759413D09EF7AE-001
[junit4] 2> Feb 01, 2018 3:27:25 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {shard_s=PostingsFormat(name=Memory), rnd_b=PostingsFormat(name=Asserting), a_t=PostingsFormat(name=Memory), id=PostingsFormat(name=Asserting)}, docValues:{shard_s=DocValuesFormat(name=Lucene70), rnd_b=DocValuesFormat(name=Memory), _version_=DocValuesFormat(name=Asserting), n_ti=DocValuesFormat(name=Lucene70), a_t=DocValuesFormat(name=Lucene70), a_i=DocValuesFormat(name=Memory), id=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1613, maxMBSortInHeap=6.918001884944582, sim=RandomSimilarity(queryNorm=false): {}, locale=bg-BG, timezone=Australia/Victoria
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=138599000,total=524288000
[junit4] 2> NOTE: All tests run in this JVM: [BlockJoinFacetDistribTest, TestReversedWildcardFilterFactory, DistributedFacetPivotSmallTest, DirectoryFactoryTest, SolrMetricManagerTest, CurrencyRangeFacetCloudTest, TestSkipOverseerOperations, AsyncCallRequestStatusResponseTest, DocumentBuilderTest, TestNumericTerms32, DistanceUnitsTest, TestRebalanceLeaders, BaseCdcrDistributedZkTest, UpdateRequestProcessorFactoryTest, TestDistributedGrouping, TestRTGBase, LegacyCloudClusterPropTest, QueryEqualityTest, TestSolr4Spatial2, TriggerIntegrationTest, QueryParsingTest, BasicZkTest, DistributedQueryComponentOptimizationTest, TestAuthorizationFramework, TestStandardQParsers, BigEndianAscendingWordDeserializerTest, RestartWhileUpdatingTest, BasicAuthIntegrationTest, TestDocBasedVersionConstraints, TimeRoutedAliasUpdateProcessorTest, TestScoreJoinQPNoScore, SearchHandlerTest, TestSolrIndexConfig, StressHdfsTest, ZkShardTermsTest, HdfsLockFactoryTest, TestLeaderInitiatedRecoveryThread, SpatialHeatmapFacetsTest, TestConfigSetsAPIExclusivity, TestConfigSetImmutable, ConfigSetsAPITest, TestQueryUtils, MultiThreadedOCPTest, TestJettySolrRunner, PropertiesRequestHandlerTest, TestCoreBackup, SolrJmxReporterTest, SaslZkACLProviderTest, TestSchemaManager, TestFilteredDocIdSet, PeerSyncTest, TestSha256AuthenticationProvider, UninvertDocValuesMergePolicyTest, TestMacroExpander, TestSegmentSorting, TestSolrCloudWithSecureImpersonation, DistributedFacetPivotLargeTest, TestSubQueryTransformerDistrib, TestFieldCacheVsDocValues, TestDynamicFieldCollectionResource, TestDeleteCollectionOnDownNodes, TestJsonFacetRefinement, SchemaApiFailureTest, TestCloudSearcherWarming, SearchRateTriggerTest, HdfsNNFailoverTest, SampleTest, TestDistributedStatsComponentCardinality, TestExportWriter, TestUseDocValuesAsStored2, TestOrdValues, PeerSyncReplicationTest, TestManagedStopFilterFactory, TestElisionMultitermQuery, TestFuzzyAnalyzedSuggestions, ExecutePlanActionTest, TestSolrCloudSnapshots, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestLargeCluster, TermsComponentTest, SuggesterFSTTest, PrimUtilsTest, DistributedMLTComponentTest, DeleteStatusTest, TestXmlQParser, HdfsBasicDistributedZkTest, TestLRUCache, OverseerRolesTest, TestExactSharedStatsCache, AddBlockUpdateTest, ExplicitHLLTest, TestQueryWrapperFilter, CoreMergeIndexesAdminHandlerTest, TestPostingsSolrHighlighter, SolrCloudExampleTest, TestSearchPerf, TestExclusionRuleCollectionAccess, ZkControllerTest, SolrInfoBeanTest, TestFreeTextSuggestions, SoftAutoCommitTest, CachingDirectoryFactoryTest, TestPivotHelperCode, FileUtilsTest, DistributedDebugComponentTest, SimplePostToolTest, SolrShardReporterTest, TestSolrJ, SolrCoreTest, TestAnalyzeInfixSuggestions, TestReload, TestPullReplicaErrorHandling, TestSchemaNameResource, AddReplicaTest, TestCollectionsAPIViaSolrCloudCluster, ClassificationUpdateProcessorTest, BlockDirectoryTest, TestSQLHandlerNonCloud, TestStressRecovery, TestConfigsApi, TestTrie, TestTolerantUpdateProcessorRandomCloud, TestImplicitCoreProperties, HttpPartitionTest, CSVRequestHandlerTest, OverriddenZkACLAndCredentialsProvidersTest, DistributedIntervalFacetingTest, CollectionStateFormat2Test, ConcurrentCreateRoutedAliasTest, ForceLeaderTest, TestCloudRecovery, TestLeaderElectionWithEmptyReplica, TestPrepRecovery, TestStressCloudBlindAtomicUpdates, TestStressLiveNodes, AssignTest, ShardSplitTest]
[junit4] Completed [623/773 (10!)] on J1 in 763.97s, 5 tests, 1 failure <<< FAILURES!
[...truncated 48726 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 345 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/345/
3 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTriggerRestoreState
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([70E0E3CC850E755D:F8DD6AB3BFCE94F0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTriggerRestoreState(TriggerIntegrationTest.java:426)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testCooldown
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([70E0E3CC850E755D:415E8E28FBA400AF]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testCooldown(TriggerIntegrationTest.java:1177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestDistribStateManager.testGetSetRemoveData
Error Message:
Node watch should have fired!
Stack Trace:
java.lang.AssertionError: Node watch should have fired!
at __randomizedtesting.SeedInfo.seed([70E0E3CC850E755D:56700C9ED180B357]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestDistribStateManager.testGetSetRemoveData(TestDistribStateManager.java:256)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12285 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestDistribStateManager
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestDistribStateManager_70E0E3CC850E755D-001/init-core-data-001
[junit4] 2> 1063171 INFO (SUITE-TestDistribStateManager-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1063172 INFO (SUITE-TestDistribStateManager-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 1063173 INFO (SUITE-TestDistribStateManager-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1063186 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateMode
[junit4] 2> 1063187 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1063187 INFO (Thread-1258) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1063187 INFO (Thread-1258) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1063214 ERROR (Thread-1258) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1063287 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkTestServer start zk server on port:46397
[junit4] 2> 1063289 INFO (zkConnectionManagerCallback-1161-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1063290 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 1063344 INFO (zkConnectionManagerCallback-1163-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1063344 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 1063347 ERROR (TEST-TestDistribStateManager.testCreateMode-seed#[70E0E3CC850E755D]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1063364 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46397 46397
[junit4] 2> 1063678 INFO (Thread-1258) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46397 46397
[junit4] 2> 1063680 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateMode
[junit4] 2> 1063700 INFO (TEST-TestDistribStateManager.testMulti-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMulti
[junit4] 2> 1063703 INFO (TEST-TestDistribStateManager.testMulti-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.cloud.autoscaling.sim.SimDistribStateManager
[junit4] 2> 1063703 INFO (TEST-TestDistribStateManager.testMulti-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testMulti
[junit4] 2> 1063719 INFO (TEST-TestDistribStateManager.testListData-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testListData
[junit4] 2> 1063719 INFO (TEST-TestDistribStateManager.testListData-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.cloud.autoscaling.sim.SimDistribStateManager
[junit4] 2> 1063720 INFO (TEST-TestDistribStateManager.testListData-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testListData
[junit4] 2> 1063733 INFO (TEST-TestDistribStateManager.testHasData-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testHasData
[junit4] 2> 1063733 INFO (TEST-TestDistribStateManager.testHasData-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1063734 INFO (Thread-1259) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1063734 INFO (Thread-1259) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1063749 ERROR (Thread-1259) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1063834 INFO (TEST-TestDistribStateManager.testHasData-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkTestServer start zk server on port:45680
[junit4] 2> 1063836 INFO (zkConnectionManagerCallback-1165-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1063836 INFO (TEST-TestDistribStateManager.testHasData-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 1063840 ERROR (TEST-TestDistribStateManager.testHasData-seed#[70E0E3CC850E755D]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1063854 INFO (TEST-TestDistribStateManager.testHasData-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45680 45680
[junit4] 2> 1063915 INFO (Thread-1259) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45680 45680
[junit4] 2> 1063916 INFO (TEST-TestDistribStateManager.testHasData-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testHasData
[junit4] 2> 1063918 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testGetSetRemoveData
[junit4] 2> 1063918 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1063918 INFO (Thread-1260) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1063918 INFO (Thread-1260) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1063919 ERROR (Thread-1260) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1064020 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkTestServer start zk server on port:38692
[junit4] 2> 1064045 INFO (zkConnectionManagerCallback-1167-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1064046 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 1064050 INFO (zkConnectionManagerCallback-1169-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1064094 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 1069096 ERROR (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[70E0E3CC850E755D]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1069096 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:38692 38692
[junit4] 2> 1069326 INFO (Thread-1260) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:38692 38692
[junit4] 2> 1069326 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testGetSetRemoveData
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDistribStateManager -Dtests.method=testGetSetRemoveData -Dtests.seed=70E0E3CC850E755D -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=hr -Dtests.timezone=America/Noronha -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 5.41s J1 | TestDistribStateManager.testGetSetRemoveData <<<
[junit4] > Throwable #1: java.lang.AssertionError: Node watch should have fired!
[junit4] > at __randomizedtesting.SeedInfo.seed([70E0E3CC850E755D:56700C9ED180B357]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestDistribStateManager.testGetSetRemoveData(TestDistribStateManager.java:256)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestDistribStateManager_70E0E3CC850E755D-001
[junit4] 2> Feb 01, 2018 12:24:55 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1248, maxMBSortInHeap=7.3207745729132885, sim=RandomSimilarity(queryNorm=true): {}, locale=hr, timezone=America/Noronha
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=182314232,total=517996544
[junit4] 2> NOTE: All tests run in this JVM: [TestMacroExpander, DistributedDebugComponentTest, TestPullReplicaErrorHandling, SmileWriterTest, TestLegacyField, TestRebalanceLeaders, DeleteStatusTest, SolrCloudExampleTest, ReplaceNodeNoTargetTest, TestStandardQParsers, TestXmlQParser, TestSchemaVersionResource, LegacyCloudClusterPropTest, DistributedQueryComponentOptimizationTest, HttpPartitionTest, TestObjectReleaseTracker, HLLUtilTest, RestartWhileUpdatingTest, DistributedFacetPivotLargeTest, BasicFunctionalityTest, TestSolrCloudSnapshots, HdfsNNFailoverTest, TestSolr4Spatial, SuggesterFSTTest, TestSubQueryTransformerDistrib, TestInitQParser, BadComponentTest, ExplicitHLLTest, TestDFISimilarityFactory, ClassificationUpdateProcessorTest, TestPivotHelperCode, HighlighterTest, DistribJoinFromCollectionTest, AlternateDirectoryTest, RulesTest, TestFunctionQuery, UpdateRequestProcessorFactoryTest, GraphQueryTest, EnumFieldTest, TriLevelCompositeIdRoutingTest, TestCorePropertiesReload, SuggesterTest, PeerSyncTest, CachingDirectoryFactoryTest, TestLeaderInitiatedRecoveryThread, TestOverriddenPrefixQueryForCustomFieldType, DistributedIntervalFacetingTest, DistributedMLTComponentTest, ScriptEngineTest, TestAnalyzeInfixSuggestions, PrimitiveFieldTypeTest, ConfigSetsAPITest, FastVectorHighlighterTest, SearchHandlerTest, PolyFieldTest, TestSolrJ, TestNumericRangeQuery64, AddBlockUpdateTest, TestSolrCloudWithHadoopAuthPlugin, QueryElevationComponentTest, TestPerFieldSimilarity, OverseerRolesTest, TestDistribStateManager]
[junit4] Completed [185/773 (1!)] on J1 in 7.07s, 5 tests, 1 failure <<< FAILURES!
[...truncated 1524 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_70E0E3CC850E755D-001/init-core-data-001
[junit4] 2> 3611264 INFO (SUITE-TriggerIntegrationTest-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 3611275 INFO (SUITE-TriggerIntegrationTest-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 3611275 INFO (SUITE-TriggerIntegrationTest-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 3611276 INFO (SUITE-TriggerIntegrationTest-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_70E0E3CC850E755D-001/tempDir-001
[junit4] 2> 3611276 INFO (SUITE-TriggerIntegrationTest-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3611282 INFO (Thread-8143) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3611282 INFO (Thread-8143) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3611291 ERROR (Thread-8143) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3611382 INFO (SUITE-TriggerIntegrationTest-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46437
[junit4] 2> 3611386 INFO (zkConnectionManagerCallback-5162-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611392 INFO (jetty-launcher-5159-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T11:27:37+14:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3611392 INFO (jetty-launcher-5159-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T11:27:37+14:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3611393 INFO (jetty-launcher-5159-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3611393 INFO (jetty-launcher-5159-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3611393 INFO (jetty-launcher-5159-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3611393 INFO (jetty-launcher-5159-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 3611393 INFO (jetty-launcher-5159-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3611393 INFO (jetty-launcher-5159-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 3611393 INFO (jetty-launcher-5159-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7eb3a45d{/solr,null,AVAILABLE}
[junit4] 2> 3611394 INFO (jetty-launcher-5159-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3e47f1fe{/solr,null,AVAILABLE}
[junit4] 2> 3611395 INFO (jetty-launcher-5159-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2f9ce966{SSL,[ssl, http/1.1]}{127.0.0.1:44315}
[junit4] 2> 3611395 INFO (jetty-launcher-5159-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@72026bf0{SSL,[ssl, http/1.1]}{127.0.0.1:37754}
[junit4] 2> 3611395 INFO (jetty-launcher-5159-thread-1) [ ] o.e.j.s.Server Started @3621696ms
[junit4] 2> 3611395 INFO (jetty-launcher-5159-thread-2) [ ] o.e.j.s.Server Started @3621696ms
[junit4] 2> 3611395 INFO (jetty-launcher-5159-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44315}
[junit4] 2> 3611395 INFO (jetty-launcher-5159-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37754}
[junit4] 2> 3611396 ERROR (jetty-launcher-5159-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3611396 ERROR (jetty-launcher-5159-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3611396 INFO (jetty-launcher-5159-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 3611396 INFO (jetty-launcher-5159-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 3611396 INFO (jetty-launcher-5159-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3611396 INFO (jetty-launcher-5159-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3611396 INFO (jetty-launcher-5159-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3611396 INFO (jetty-launcher-5159-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3611396 INFO (jetty-launcher-5159-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-01T01:07:16.236Z
[junit4] 2> 3611396 INFO (jetty-launcher-5159-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-01T01:07:16.236Z
[junit4] 2> 3611397 INFO (zkConnectionManagerCallback-5165-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611397 INFO (zkConnectionManagerCallback-5166-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611397 INFO (jetty-launcher-5159-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3611397 INFO (jetty-launcher-5159-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3611404 INFO (jetty-launcher-5159-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46437/solr
[junit4] 2> 3611404 INFO (jetty-launcher-5159-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46437/solr
[junit4] 2> 3611408 INFO (zkConnectionManagerCallback-5174-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611408 INFO (zkConnectionManagerCallback-5172-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611410 INFO (zkConnectionManagerCallback-5176-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611411 INFO (zkConnectionManagerCallback-5178-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611469 INFO (jetty-launcher-5159-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3611469 INFO (jetty-launcher-5159-thread-1) [n:127.0.0.1:44315_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3611469 INFO (jetty-launcher-5159-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37754_solr
[junit4] 2> 3611470 INFO (jetty-launcher-5159-thread-1) [n:127.0.0.1:44315_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44315_solr
[junit4] 2> 3611470 INFO (jetty-launcher-5159-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.Overseer Overseer (id=72182215205978118-127.0.0.1:37754_solr-n_0000000000) starting
[junit4] 2> 3611472 INFO (zkCallback-5175-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3611474 INFO (zkCallback-5177-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3611479 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 3611480 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 3611480 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:44315_solr]
[junit4] 2> 3611480 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 3611480 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 3611480 INFO (jetty-launcher-5159-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37754_solr
[junit4] 2> 3611491 INFO (zkCallback-5175-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3611491 INFO (zkCallback-5177-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3611495 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3611496 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 3611496 DEBUG (ScheduledTrigger-8679-thread-1) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3611515 INFO (jetty-launcher-5159-thread-1) [n:127.0.0.1:44315_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44315.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3611521 INFO (jetty-launcher-5159-thread-1) [n:127.0.0.1:44315_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44315.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3611521 INFO (jetty-launcher-5159-thread-1) [n:127.0.0.1:44315_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44315.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3611522 INFO (jetty-launcher-5159-thread-1) [n:127.0.0.1:44315_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_70E0E3CC850E755D-001/tempDir-001/node1/.
[junit4] 2> 3611535 INFO (jetty-launcher-5159-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37754.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3611542 INFO (jetty-launcher-5159-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37754.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3611542 INFO (jetty-launcher-5159-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37754.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3611543 INFO (jetty-launcher-5159-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_70E0E3CC850E755D-001/tempDir-001/node2/.
[junit4] 2> 3611560 INFO (zkConnectionManagerCallback-5184-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611562 INFO (zkConnectionManagerCallback-5188-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611563 INFO (SUITE-TriggerIntegrationTest-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3611563 INFO (SUITE-TriggerIntegrationTest-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46437/solr ready
[junit4] 2> 3611624 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
[junit4] 2> 3611624 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 3611624 DEBUG (zkCallback-5177-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 3611624 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 3611626 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3611626 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3611627 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 3611644 INFO (qtp1474490809-26991) [n:127.0.0.1:37754_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3611648 INFO (qtp1474490809-26991) [n:127.0.0.1:37754_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 3611674 INFO (qtp1474490809-26997) [n:127.0.0.1:37754_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 3611676 INFO (qtp798780450-26983) [n:127.0.0.1:44315_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 3611676 DEBUG (qtp798780450-26992) [n:127.0.0.1:44315_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3611677 INFO (qtp798780450-26992) [n:127.0.0.1:44315_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=7
[junit4] 2> 3611677 DEBUG (zkCallback-5177-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 3611678 INFO (zkCallback-5177-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventQueueAction instantiated
[junit4] 2> 3611680 DEBUG (zkCallback-5177-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44315_solr, 127.0.0.1:37754_solr]
[junit4] 2> 3611680 DEBUG (zkCallback-5177-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEQ instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestEventQueueAction}], enabled=true}
[junit4] 2> 3611680 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 3611681 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 3611681 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3611682 DEBUG (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 3611682 DEBUG (ScheduledTrigger-8679-thread-1) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 3611682 DEBUG (ScheduledTrigger-8679-thread-1) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3611682 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T11:27:37+14:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3611683 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3611683 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3611683 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 3611683 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2707223f{/solr,null,AVAILABLE}
[junit4] 2> 3611684 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6b45886f{SSL,[ssl, http/1.1]}{127.0.0.1:44545}
[junit4] 2> 3611684 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.Server Started @3621984ms
[junit4] 2> 3611684 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44545}
[junit4] 2> 3611684 ERROR (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3611684 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 3611684 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3611684 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3611684 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-01T01:07:16.524Z
[junit4] 2> 3611685 INFO (zkConnectionManagerCallback-5190-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611686 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3611695 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46437/solr
[junit4] 2> 3611695 INFO (zkConnectionManagerCallback-5194-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611702 INFO (zkConnectionManagerCallback-5196-thread-1-processing-n:127.0.0.1:44545_solr) [n:127.0.0.1:44545_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3611706 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [n:127.0.0.1:44545_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3611707 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [n:127.0.0.1:44545_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3611708 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [n:127.0.0.1:44545_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44545_solr
[junit4] 2> 3611709 INFO (zkCallback-5175-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3611711 INFO (zkCallback-5195-thread-1-processing-n:127.0.0.1:44545_solr) [n:127.0.0.1:44545_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3611721 INFO (zkCallback-5177-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3611722 INFO (zkCallback-5187-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3611764 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [n:127.0.0.1:44545_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44545.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3611781 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [n:127.0.0.1:44545_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44545.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3611781 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [n:127.0.0.1:44545_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44545.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3611783 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [n:127.0.0.1:44545_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_70E0E3CC850E755D-001/tempDir-001/node3/.
[junit4] 2> 3612686 DEBUG (ScheduledTrigger-8679-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 3612686 DEBUG (ScheduledTrigger-8679-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3612686 DEBUG (ScheduledTrigger-8679-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:44545_solr at time 1901569519320770
[junit4] 2> 3613647 INFO (OverseerCollectionConfigSetProcessor-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3613687 DEBUG (ScheduledTrigger-8679-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 3613687 DEBUG (ScheduledTrigger-8679-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 3613687 DEBUG (ScheduledTrigger-8679-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEQ firing registered processor for nodes: [127.0.0.1:44545_solr] added at times [1901569519320770], now=1901570520107840
[junit4] 2> 3613687 DEBUG (ScheduledTrigger-8679-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6c177a7b6fec2T19eb92hubht45ylnjfpt46qfn",
[junit4] 2> "source":"node_added_triggerEQ",
[junit4] 2> "eventTime":1901569519320770,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1901569519320770],
[junit4] 2> "nodeNames":["127.0.0.1:44545_solr"]}}
[junit4] 2> 3613689 DEBUG (ScheduledTrigger-8679-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_triggerEQ]
[junit4] 2> 3613689 INFO (ScheduledTrigger-8679-thread-2) [n:127.0.0.1:37754_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 3613690 DEBUG (AutoscalingActionExecutor-8680-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6c177a7b6fec2T19eb92hubht45ylnjfpt46qfn",
[junit4] 2> "source":"node_added_triggerEQ",
[junit4] 2> "eventTime":1901569519320770,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1901569519320770],
[junit4] 2> "_enqueue_time_":1901570522406197,
[junit4] 2> "nodeNames":["127.0.0.1:44545_solr"]}}
[junit4] 2> 3613690 INFO (AutoscalingActionExecutor-8680-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.a.TriggerIntegrationTest -- event: {
[junit4] 2> "id":"6c177a7b6fec2T19eb92hubht45ylnjfpt46qfn",
[junit4] 2> "source":"node_added_triggerEQ",
[junit4] 2> "eventTime":1901569519320770,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1901569519320770],
[junit4] 2> "_enqueue_time_":1901570522406197,
[junit4] 2> "nodeNames":["127.0.0.1:44545_solr"]}}
[junit4] 2> 3613691 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@72026bf0{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 3613692 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1993029186
[junit4] 2> 3613692 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 3613692 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2eabfafc: rootName = solr_37754, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@610edddc
[junit4] 2> 3613698 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 3613698 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@72b4bae3: rootName = solr_37754, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@96faf96
[junit4] 2> 3613702 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 3613702 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7cbdec38: rootName = solr_37754, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6b77d146
[junit4] 2> 3613702 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37754_solr
[junit4] 2> 3613703 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 3613703 INFO (zkCallback-5195-thread-1-processing-n:127.0.0.1:44545_solr) [n:127.0.0.1:44545_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 3613703 INFO (zkCallback-5187-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 3613703 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3613703 INFO (zkCallback-5175-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 3613703 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.Overseer Overseer (id=72182215205978118-127.0.0.1:37754_solr-n_0000000000) closing
[junit4] 2> 3613703 INFO (OverseerStateUpdate-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37754_solr
[junit4] 2> 3613703 INFO (zkCallback-5177-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 3613705 WARN (OverseerAutoScalingTriggerThread-72182215205978118-127.0.0.1:37754_solr-n_0000000000) [n:127.0.0.1:37754_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3613706 WARN (AutoscalingActionExecutor-8680-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_triggerEQ has been closed.
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.dequeue(ScheduledTriggers.java:526)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:310)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3613705 DEBUG (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3613710 DEBUG (AutoscalingActionExecutor-8680-thread-1-processing-n:127.0.0.1:37754_solr) [n:127.0.0.1:37754_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 20 ms for event id=6c177a7b6fec2T19eb92hubht45ylnjfpt46qfn
[junit4] 2> 3613711 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3e47f1fe{/solr,null,UNAVAILABLE}
[junit4] 2> 3613711 INFO (zkCallback-5175-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44315_solr
[junit4] 2> 3613712 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3613713 INFO (zkCallback-5175-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.Overseer Overseer (id=72182215205978117-127.0.0.1:44315_solr-n_0000000001) starting
[junit4] 2> 3613716 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 3613716 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 3613716 INFO (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventQueueAction instantiated
[junit4] 2> 3613716 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44315_solr, 127.0.0.1:44545_solr]
[junit4] 2> 3613716 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEQ instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestEventQueueAction}], enabled=true}
[junit4] 2> 3613717 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:44315_solr, 127.0.0.1:44545_solr]
[junit4] 2> 3613717 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion -1
[junit4] 2> 3613717 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 3613718 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 3613718 DEBUG (ScheduledTrigger-8694-thread-1) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3613718 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 3613719 DEBUG (ScheduledTrigger-8694-thread-2) [n:127.0.0.1:44315_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6c177a7b6fec2T19eb92hubht45ylnjfpt46qfn",
[junit4] 2> "source":"node_added_triggerEQ",
[junit4] 2> "eventTime":1901569519320770,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1901569519320770],
[junit4] 2> "_dequeue_time_":1901570552020534,
[junit4] 2> "_enqueue_time_":1901570522406197,
[junit4] 2> "nodeNames":["127.0.0.1:44545_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 3613720 DEBUG (ScheduledTrigger-8694-thread-2) [n:127.0.0.1:44315_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_triggerEQ]
[junit4] 2> 3613720 INFO (ScheduledTrigger-8694-thread-2) [n:127.0.0.1:44315_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 3613720 DEBUG (AutoscalingActionExecutor-8695-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6c177a7b6fec2T19eb92hubht45ylnjfpt46qfn",
[junit4] 2> "source":"node_added_triggerEQ",
[junit4] 2> "eventTime":1901569519320770,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1901569519320770],
[junit4] 2> "_dequeue_time_":1901570552020534,
[junit4] 2> "_enqueue_time_":1901570522406197,
[junit4] 2> "nodeNames":["127.0.0.1:44545_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 3613720 INFO (AutoscalingActionExecutor-8695-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.a.TriggerIntegrationTest -- event: {
[junit4] 2> "id":"6c177a7b6fec2T19eb92hubht45ylnjfpt46qfn",
[junit4] 2> "source":"node_added_triggerEQ",
[junit4] 2> "eventTime":1901569519320770,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1901569519320770],
[junit4] 2> "_dequeue_time_":1901570552020534,
[junit4] 2> "_enqueue_time_":1901570522406197,
[junit4] 2> "nodeNames":["127.0.0.1:44545_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 3613721 DEBUG (ScheduledTrigger-8694-thread-2) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 3613721 DEBUG (ScheduledTrigger-8694-thread-2) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3613722 DEBUG (AutoscalingActionExecutor-8695-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms
[junit4] 2> 3613722 DEBUG (AutoscalingActionExecutor-8695-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerEQ after 5000ms
[junit4] 2> 3613722 DEBUG (AutoscalingActionExecutor-8695-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=6c177a7b6fec2T19eb92hubht45ylnjfpt46qfn
[junit4] 2> 3618713 INFO (TEST-TriggerIntegrationTest.testEventQueue-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEventQueue
[junit4] 2> 3618722 DEBUG (ScheduledTrigger-8694-thread-1) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3618724 DEBUG (ScheduledTrigger-8694-thread-3) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEQ
[junit4] 2> 3618725 DEBUG (ScheduledTrigger-8694-thread-3) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3618776 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testContinueTriggersOnOverseerRestart
[junit4] 2> 3618776 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 5
[junit4] 2> 3618776 DEBUG (zkCallback-5175-thread-1-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 3618777 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 3618779 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3618779 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 3618779 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 3618781 INFO (qtp798780450-26988) [n:127.0.0.1:44315_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3618788 INFO (qtp798780450-26988) [n:127.0.0.1:44315_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=7
[junit4] 2> 3618797 INFO (qtp798780450-26994) [n:127.0.0.1:44315_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 3618826 INFO (qtp1425344267-27051) [n:127.0.0.1:44545_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 3618827 DEBUG (qtp798780450-26992) [n:127.0.0.1:44315_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 3618827 INFO (qtp798780450-26992) [n:127.0.0.1:44315_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=32
[junit4] 2> 3618827 DEBUG (zkCallback-5175-thread-2-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 3618829 DEBUG (zkCallback-5175-thread-2-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44315_solr, 127.0.0.1:44545_solr]
[junit4] 2> 3618829 DEBUG (zkCallback-5175-thread-2-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerCTOOR instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 3618829 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 3618830 INFO (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 3618831 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 3618831 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@2f9ce966{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 3618832 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1089204499
[junit4] 2> 3618833 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 3618833 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@25e395d8: rootName = solr_44315, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@540a1e8f
[junit4] 2> 3618838 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 3618838 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1efba6e6: rootName = solr_44315, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@96faf96
[junit4] 2> 3618842 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 3618842 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1bc93131: rootName = solr_44315, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6b77d146
[junit4] 2> 3618842 DEBUG (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 3618842 DEBUG (ScheduledTrigger-8694-thread-4) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3618843 DEBUG (ScheduledTrigger-8694-thread-4) [n:127.0.0.1:44315_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3618843 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44315_solr
[junit4] 2> 3618844 INFO (zkCallback-5195-thread-1-processing-n:127.0.0.1:44545_solr) [n:127.0.0.1:44545_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 3618844 INFO (zkCallback-5187-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 3618844 INFO (zkCallback-5175-thread-2-processing-n:127.0.0.1:44315_solr) [n:127.0.0.1:44315_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 3618844 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 3618844 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3618844 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.Overseer Overseer (id=72182215205978117-127.0.0.1:44315_solr-n_0000000001) closing
[junit4] 2> 3618845 INFO (OverseerStateUpdate-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44315_solr
[junit4] 2> 3618845 WARN (OverseerAutoScalingTriggerThread-72182215205978117-127.0.0.1:44315_solr-n_0000000001) [n:127.0.0.1:44315_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3618847 DEBUG (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3618848 INFO (zkCallback-5195-thread-1-processing-n:127.0.0.1:44545_solr) [n:127.0.0.1:44545_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44545_solr
[junit4] 2> 3618848 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7eb3a45d{/solr,null,UNAVAILABLE}
[junit4] 2> 3618850 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3618851 INFO (zkCallback-5195-thread-1-processing-n:127.0.0.1:44545_solr) [n:127.0.0.1:44545_solr ] o.a.s.c.Overseer Overseer (id=72182215205978123-127.0.0.1:44545_solr-n_0000000002) starting
[junit4] 2> 3618854 DEBUG (OverseerAutoScalingTriggerThread-72182215205978123-127.0.0.1:44545_solr-n_0000000002) [n:127.0.0.1:44545_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 3618855 DEBUG (OverseerAutoScalingTriggerThread-72182215205978123-127.0.0.1:44545_solr-n_0000000002) [n:127.0.0.1:44545_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 3618855 DEBUG (OverseerAutoScalingTriggerThread-72182215205978123-127.0.0.1:44545_solr-n_0000000002) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44545_solr]
[junit4] 2> 3618856 DEBUG (OverseerAutoScalingTriggerThread-72182215205978123-127.0.0.1:44545_solr-n_0000000002) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerCTOOR instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 3618856 DEBUG (OverseerAutoScalingTriggerThread-72182215205978123-127.0.0.1:44545_solr-n_0000000002) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:44545_solr]
[junit4] 2> 3618856 DEBUG (OverseerAutoScalingTriggerThread-72182215205978123-127.0.0.1:44545_solr-n_0000000002) [n:127.0.0.1:44545_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 3618856 DEBUG (OverseerAutoScalingTriggerThread-72182215205978123-127.0.0.1:44545_solr-n_0000000002) [n:127.0.0.1:44545_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 3618856 INFO (OverseerAutoScalingTriggerThread-72182215205978123-127.0.0.1:44545_solr-n_0000000002) [n:127.0.0.1:44545_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 3618858 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3618858 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 3618858 DEBUG (OverseerAutoScalingTriggerThread-72182215205978123-127.0.0.1:44545_solr-n_0000000002) [n:127.0.0.1:44545_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 3618858 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3619858 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3619858 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 3619862 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3620862 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3620862 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 3620862 DEBUG (ScheduledTrigger-8699-thread-4) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3621862 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3621862 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 3621863 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3622862 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3622863 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 3622863 DEBUG (ScheduledTrigger-8699-thread-4) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3623863 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3623863 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 3623863 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3624863 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3624863 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 3624863 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3625863 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3625863 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3625864 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 3626864 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3626864 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3626864 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 3627864 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3627864 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3627864 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 3628852 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T11:27:37+14:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3628852 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3628852 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3628852 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 3628853 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@617ec6f7{/solr,null,AVAILABLE}
[junit4] 2> 3628853 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5afd2a89{SSL,[ssl, http/1.1]}{127.0.0.1:33557}
[junit4] 2> 3628853 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.e.j.s.Server Started @3639153ms
[junit4] 2> 3628853 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33557}
[junit4] 2> 3628853 ERROR (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3628853 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 3628853 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3628854 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3628854 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-01T01:07:33.694Z
[junit4] 2> 3628855 INFO (zkConnectionManagerCallback-5200-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3628855 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3628861 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46437/solr
[junit4] 2> 3628862 INFO (zkConnectionManagerCallback-5204-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3628864 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 3628864 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3628864 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 3628865 INFO (zkConnectionManagerCallback-5206-thread-1-processing-n:127.0.0.1:33557_solr) [n:127.0.0.1:33557_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3628869 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [n:127.0.0.1:33557_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3628870 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [n:127.0.0.1:33557_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3628871 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [n:127.0.0.1:33557_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33557_solr
[junit4] 2> 3628872 INFO (zkCallback-5195-thread-1-processing-n:127.0.0.1:44545_solr) [n:127.0.0.1:44545_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3628872 INFO (zkCallback-5187-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3628874 INFO (zkCallback-5205-thread-1-processing-n:127.0.0.1:33557_solr) [n:127.0.0.1:33557_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3628928 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [n:127.0.0.1:33557_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33557.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3628936 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [n:127.0.0.1:33557_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33557.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3628937 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [n:127.0.0.1:33557_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33557.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19aad4dd
[junit4] 2> 3628938 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[70E0E3CC850E755D]) [n:127.0.0.1:33557_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_70E0E3CC850E755D-001/tempDir-001/node4/.
[junit4] 2> 3629864 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3629865 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3629865 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3629865 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:33557_solr at time 1901586698141485
[junit4] 2> 3630865 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3630865 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 3630865 DEBUG (ScheduledTrigger-8699-thread-2) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 3631865 DEBUG (ScheduledTrigger-8699-thread-1) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 3631866 DEBUG (ScheduledTrigger-8699-thread-3) [n:127.0.0.1:44545_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[...truncated too long message...]
testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testMetricTrigger.shard2.leader, tag=468762600
[junit4] 2> 3807984 INFO (jetty-closer-5160-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@dd59e28{/solr,null,UNAVAILABLE}
[junit4] 2> 3807986 INFO (jetty-closer-5160-thread-5) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3807991 INFO (zkCallback-5288-thread-1-processing-n:127.0.0.1:39330_solr) [n:127.0.0.1:39330_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39330_solr
[junit4] 2> 3807991 INFO (jetty-closer-5160-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@8e4cd50{/solr,null,UNAVAILABLE}
[junit4] 2> 3807993 INFO (jetty-closer-5160-thread-4) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3809496 INFO (jetty-closer-5160-thread-7) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@22837be3{/solr,null,UNAVAILABLE}
[junit4] 2> 3809498 INFO (jetty-closer-5160-thread-7) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3809498 ERROR (SUITE-TriggerIntegrationTest-seed#[70E0E3CC850E755D]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3809500 INFO (SUITE-TriggerIntegrationTest-seed#[70E0E3CC850E755D]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46437 46437
[junit4] 2> 3809664 INFO (Thread-8143) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46437 46437
[junit4] 2> 3809668 WARN (Thread-8143) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 42 /solr/collections/collection1/terms/shard1
[junit4] 2> 19 /solr/aliases.json
[junit4] 2> 18 /solr/security.json
[junit4] 2> 10 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 9 /solr/configs/conf
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 119 /solr/collections/collection1/state.json
[junit4] 2> 32 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 19 /solr/clusterprops.json
[junit4] 2> 19 /solr/clusterstate.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 3 /solr/overseer_elect/election/72182215205978160-127.0.0.1:45661_solr-n_0000000014
[junit4] 2> 2 /solr/overseer_elect/election/72182215205978132-127.0.0.1:34362_solr-n_0000000005
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/72182215205978138-core_node6-n_0000000002
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/72182215205978141-core_node4-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/72182215205978141-127.0.0.1:41361_solr-n_0000000008
[junit4] 2> 2 /solr/overseer_elect/election/72182215205978144-127.0.0.1:36018_solr-n_0000000009
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 19 /solr/live_nodes
[junit4] 2> 19 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/queue
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_70E0E3CC850E755D-001
[junit4] 2> Feb 01, 2018 1:10:34 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=Lucene50(blocksize=128), x_s=BlockTreeOrds(blocksize=128)}, docValues:{}, maxPointsInLeafNode=1685, maxMBSortInHeap=5.211775090727315, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-KW, timezone=Europe/Rome
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=98589392,total=458227712
[junit4] 2> NOTE: All tests run in this JVM: [SpatialFilterTest, HdfsAutoAddReplicasIntegrationTest, TestManagedResource, CopyFieldTest, TestDelegationWithHadoopAuth, DisMaxRequestHandlerTest, RecoveryAfterSoftCommitTest, SolrGraphiteReporterTest, TestCloudManagedSchema, TestRealTimeGet, TestAtomicUpdateErrorCases, TestStressReorder, TestFieldCacheSort, ClusterStateTest, TestSimDistributedQueue, TestBinaryResponseWriter, SimpleMLTQParserTest, TestCloudNestedDocsSort, MigrateRouteKeyTest, VMParamsZkACLAndCredentialsProvidersTest, ConnectionReuseTest, AutoScalingHandlerTest, DirectSolrSpellCheckerTest, CoreAdminCreateDiscoverTest, TestCustomSort, TestBlobHandler, TestTlogReplica, WordBreakSolrSpellCheckerTest, TestUtils, TestCustomStream, TestSolrDeletionPolicy2, TestClusterStateMutator, AssignTest, OverseerCollectionConfigSetProcessorTest, ConnectionManagerTest, TestManagedSynonymFilterFactory, TestDistribDocBasedVersion, PingRequestHandlerTest, BufferStoreTest, TestRequestStatusCollectionAPI, SharedFSAutoReplicaFailoverTest, AliasIntegrationTest, TestRestoreCore, TestLockTree, ParsingFieldUpdateProcessorsTest, TestBackupRepositoryFactory, SolrMetricReporterTest, ImplicitSnitchTest, TestStressCloudBlindAtomicUpdates, TestMacros, SolrTestCaseJ4Test, TestNodeAddedTrigger, TestLegacyNumericRangeQueryBuilder, TestCodecSupport, TestClusterStateProvider, TestDefaultStatsCache, LukeRequestHandlerTest, TestPrepRecovery, NumericFieldsTest, DistribCursorPagingTest, TestRecoveryHdfs, TolerantUpdateProcessorTest, TestFaceting, TestZkAclsWithHadoopAuth, TestScoreJoinQPScore, TestPayloadCheckQParserPlugin, CursorMarkTest, SecurityConfHandlerTest, TestValueSourceCache, SolrXmlInZkTest, TestClusterProperties, TestPushWriter, TestPKIAuthenticationPlugin, NumberUtilsTest, TestFastWriter, TestSolrConfigHandler, RegexBoostProcessorTest, TestHalfAndHalfDocValues, ReplaceNodeTest, DistributedFacetPivotWhiteBoxTest, TestBulkSchemaAPI, DistributedFacetPivotLongTailTest, HdfsChaosMonkeyNothingIsSafeTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestFieldCacheWithThreads, CoreAdminHandlerTest, TestSolrConfigHandlerCloud, MinimalSchemaTest, TestNonDefinedSimilarityFactory, TestSolrCoreProperties, TestNumericTokenStream, TestLMJelinekMercerSimilarityFactory, RankQueryTest, SolrCmdDistributorTest, TestNumericTerms64, TestApiFramework, SystemLogListenerTest, SearchHandlerTest, TestGraphTermsQParserPlugin, TestExpandComponent, CacheHeaderTest, StatelessScriptUpdateProcessorFactoryTest, TestSuggestSpellingConverter, CollectionsAPIAsyncDistributedZkTest, TestMultiWordSynonyms, TestReloadAndDeleteDocs, UnloadDistributedZkTest, DistributedSuggestComponentTest, SparseHLLTest, ReplicaListTransformerTest, DateMathParserTest, LIRRollingUpdatesTest, RollingRestartTest, TestJmxIntegration, ForceLeaderTest, SuggesterTSTTest, TestSerializedLuceneMatchVersion, UniqFieldsUpdateProcessorFactoryTest, TestBlendedInfixSuggestions, TestXmlQParserPlugin, DeleteInactiveReplicaTest, TestDynamicFieldResource, RegexBytesRefFilterTest, AnalysisAfterCoreReloadTest, DocValuesMultiTest, RecoveryZkTest, TestExecutePlanAction, MoveReplicaHDFSTest, TestDistributedSearch, SpellCheckComponentTest, BadIndexSchemaTest, TestFiltering, TestIndexSearcher, ShowFileRequestHandlerTest, SimplePostToolTest, TestCoreDiscovery, SpellCheckCollatorTest, TestStressLucene, TestTrie, TestCSVLoader, NoCacheHeaderTest, TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, QueryEqualityTest, DefaultValueUpdateProcessorTest, SortByFunctionTest, DistanceFunctionTest, SolrInfoBeanTest, IndexBasedSpellCheckerTest, TestWriterPerf, FileBasedSpellCheckerTest, RequestHandlersTest, DocumentBuilderTest, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, MoreLikeThisHandlerTest, RequiredFieldsTest, IndexSchemaRuntimeFieldTest, ReturnFieldsTest, JsonLoaderTest, TestPartialUpdateDeduplication, SOLR749Test, TestQuerySenderListener, UpdateParamsTest, ResponseLogComponentTest, SolrIndexConfigTest, MultiTermTest, TestBinaryField, TestSearchPerf, TestElisionMultitermQuery, TestConfig, TestFuzzyAnalyzedSuggestions, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, SpellPossibilityIteratorTest, SynonymTokenizerTest, TestXIncludeConfig, TestDFRSimilarityFactory, TestBM25SimilarityFactory, ResourceLoaderTest, URLClassifyProcessorTest, PluginInfoTest, DateFieldTest, DOMUtilTest, TestLRUCache, TestCrossCoreJoin, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, ActionThrottleTest, AddReplicaTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, DeleteReplicaTest, LeaderInitiatedRecoveryOnShardRestartTest, MoveReplicaHDFSFailoverTest, TestCloudInspectUtil, TestCloudJSONFacetJoinDomain, TestCloudSearcherWarming, TestMiniSolrCloudClusterSSL, TestSolrCloudWithDelegationTokens, TestSolrCloudWithKerberosAlt, TlogReplayBufferedWhileIndexingTest, ZkShardTermsTest, CustomCollectionTest, SimpleCollectionCreateDeleteTest, TestCollectionAPI, TestReplicaProperties, ComputePlanActionTest, ExecutePlanActionTest, HttpTriggerListenerTest, MetricTriggerTest, NodeLostTriggerTest, TriggerIntegrationTest]
[junit4] Completed [620/773 (2!)] on J2 in 198.96s, 14 tests, 2 failures <<< FAILURES!
[...truncated 48645 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 344 - Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/344/
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([B8A6057108377028:E5EE1BF8C7F1D667]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11878 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestLargeCluster_B8A6057108377028-001/init-core-data-001
[junit4] 2> 520651 WARN (SUITE-TestLargeCluster-seed#[B8A6057108377028]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
[junit4] 2> 520651 INFO (SUITE-TestLargeCluster-seed#[B8A6057108377028]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 520652 INFO (SUITE-TestLargeCluster-seed#[B8A6057108377028]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 520653 INFO (SUITE-TestLargeCluster-seed#[B8A6057108377028]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 520730 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 520730 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 520731 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10002_solr, 127.0.0.1:10012_solr, 127.0.0.1:10007_solr, 127.0.0.1:10006_solr, 127.0.0.1:10010_solr, 127.0.0.1:10005_solr, 127.0.0.1:10009_solr, 127.0.0.1:10013_solr, 127.0.0.1:10000_solr, 127.0.0.1:10008_solr, 127.0.0.1:10003_solr, 127.0.0.1:10011_solr]
[junit4] 2> 520731 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 520731 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 520818 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 520818 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 520826 DEBUG (ScheduledTrigger-972-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 520875 DEBUG (ScheduledTrigger-972-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 520877 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
[junit4] 2> 520877 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 520877 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 520877 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 520878 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 520878 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 520894 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 520895 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:166)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 520895 DEBUG (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 520918 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 520918 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 520919 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 520919 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 520919 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 520919 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 520919 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 520919 DEBUG (ScheduledTrigger-975-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 520951 DEBUG (ScheduledTrigger-975-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521041 DEBUG (ScheduledTrigger-975-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521072 DEBUG (ScheduledTrigger-975-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521104 DEBUG (ScheduledTrigger-975-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521137 DEBUG (ScheduledTrigger-975-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521137 DEBUG (simCloudManagerPool-974-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 521137 DEBUG (simCloudManagerPool-974-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 521141 DEBUG (simCloudManagerPool-974-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 521141 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 521141 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 521141 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 521169 DEBUG (ScheduledTrigger-975-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521234 DEBUG (ScheduledTrigger-975-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521228 DEBUG (simCloudManagerPool-974-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 521243 DEBUG (simCloudManagerPool-974-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 521243 DEBUG (simCloudManagerPool-974-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 521243 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 521243 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 521243 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 521254 DEBUG (ScheduledTrigger-975-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521279 DEBUG (ScheduledTrigger-975-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521300 DEBUG (ScheduledTrigger-975-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521384 DEBUG (ScheduledTrigger-975-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521420 DEBUG (ScheduledTrigger-975-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521453 DEBUG (ScheduledTrigger-975-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521510 DEBUG (ScheduledTrigger-975-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521530 DEBUG (ScheduledTrigger-975-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521558 DEBUG (ScheduledTrigger-975-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521578 DEBUG (ScheduledTrigger-975-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521599 DEBUG (ScheduledTrigger-975-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521643 DEBUG (ScheduledTrigger-975-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521668 DEBUG (ScheduledTrigger-975-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521688 DEBUG (ScheduledTrigger-975-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521694 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.TestLargeCluster Ready after 16018 ms
[junit4] 2> 521713 DEBUG (ScheduledTrigger-975-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521757 DEBUG (ScheduledTrigger-975-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521789 DEBUG (ScheduledTrigger-975-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521834 DEBUG (ScheduledTrigger-975-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521858 DEBUG (ScheduledTrigger-975-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 521889 DEBUG (ScheduledTrigger-975-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"6995c50337ddeT62ebd1akbzicnr4uc86fnbf90",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1857471621856734,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10091_solr":4.0,
[junit4] 2> "127.0.0.1:10016_solr":4.0,
[junit4] 2> "127.0.0.1:10086_solr":4.0,
[junit4] 2> "127.0.0.1:10089_solr":4.0,
[junit4] 2> "127.0.0.1:10020_solr":4.0,
[junit4] 2> "127.0.0.1:10065_solr":4.0,
[junit4] 2> "127.0.0.1:10003_solr":4.0,
[junit4] 2> "127.0.0.1:10088_solr":4.0,
[junit4] 2> "127.0.0.1:10027_solr":4.0,
[junit4] 2> "127.0.0.1:10007_solr":4.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node10\":{\n \"core\":\"testSearchRate_shard1_replica_n10\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10020_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSearchRate_shard1_replica_n3\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10065_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSearchRate_shard1_replica_n1\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10003_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node8\":{\n \"core\":\"testSearchRate_shard1_replica_n8\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node7\":{\n \"core\":\"testSearchRate_shard1_replica_n7\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10016_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node4\":{\n \"core\":\"testSearchRate_shard1_replica_n4\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10086_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node5\":{\n \"core\":\"testSearchRate_shard1_replica_n5\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10089_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node9\":{\n \"core\":\"testSearchRate_shard1_replica_n9\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10088_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node6\":{\n \"core\":\"testSearchRate_shard1_replica_n6\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10027_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"testSearchRate_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10091_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}"],
[junit4] 2> "collection":{"testSearchRate":40.0},
[junit4] 2> "shard":{"testSearchRate":{"shard1":40.0}},
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}"]}}
[junit4] 2> 521919 DEBUG (ScheduledTrigger-975-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, search_rate_trigger]
[junit4] 2> 521964 DEBUG (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"6995c50337ddeT62ebd1akbzicnr4uc86fnbf90",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1857471621856734,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10091_solr":4.0,
[junit4] 2> "127.0.0.1:10016_solr":4.0,
[junit4] 2> "127.0.0.1:10086_solr":4.0,
[junit4] 2> "127.0.0.1:10089_solr":4.0,
[junit4] 2> "127.0.0.1:10020_solr":4.0,
[junit4] 2> "127.0.0.1:10065_solr":4.0,
[junit4] 2> "127.0.0.1:10003_solr":4.0,
[junit4] 2> "127.0.0.1:10088_solr":4.0,
[junit4] 2> "127.0.0.1:10027_solr":4.0,
[junit4] 2> "127.0.0.1:10007_solr":4.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node10\":{\n \"core\":\"testSearchRate_shard1_replica_n10\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10020_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSearchRate_shard1_replica_n3\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10065_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSearchRate_shard1_replica_n1\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10003_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node8\":{\n \"core\":\"testSearchRate_shard1_replica_n8\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node7\":{\n \"core\":\"testSearchRate_shard1_replica_n7\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10016_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node4\":{\n \"core\":\"testSearchRate_shard1_replica_n4\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10086_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node5\":{\n \"core\":\"testSearchRate_shard1_replica_n5\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10089_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node9\":{\n \"core\":\"testSearchRate_shard1_replica_n9\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10088_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node6\":{\n \"core\":\"testSearchRate_shard1_replica_n6\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10027_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"testSearchRate_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10091_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}"],
[junit4] 2> "collection":{"testSearchRate":40.0},
[junit4] 2> "shard":{"testSearchRate":{"shard1":40.0}},
[junit4] 2> "_enqueue_time_":1857481637015684,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}"]}}
[junit4] 2> 522026 DEBUG (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"6995c50337ddeT62ebd1akbzicnr4uc86fnbf90",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1857471621856734,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10091_solr":4.0,
[junit4] 2> "127.0.0.1:10016_solr":4.0,
[junit4] 2> "127.0.0.1:10086_solr":4.0,
[junit4] 2> "127.0.0.1:10089_solr":4.0,
[junit4] 2> "127.0.0.1:10020_solr":4.0,
[junit4] 2> "127.0.0.1:10065_solr":4.0,
[junit4] 2> "127.0.0.1:10003_solr":4.0,
[junit4] 2> "127.0.0.1:10088_solr":4.0,
[junit4] 2> "127.0.0.1:10027_solr":4.0,
[junit4] 2> "127.0.0.1:10007_solr":4.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node10\":{\n \"core\":\"testSearchRate_shard1_replica_n10\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10020_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSearchRate_shard1_replica_n3\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10065_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSearchRate_shard1_replica_n1\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10003_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node8\":{\n \"core\":\"testSearchRate_shard1_replica_n8\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node7\":{\n \"core\":\"testSearchRate_shard1_replica_n7\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10016_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node4\":{\n \"core\":\"testSearchRate_shard1_replica_n4\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10086_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node5\":{\n \"core\":\"testSearchRate_shard1_replica_n5\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10089_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node9\":{\n \"core\":\"testSearchRate_shard1_replica_n9\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10088_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node6\":{\n \"core\":\"testSearchRate_shard1_replica_n6\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10027_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"testSearchRate_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10091_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}"],
[junit4] 2> "collection":{"testSearchRate":40.0},
[junit4] 2> "shard":{"testSearchRate":{"shard1":40.0}},
[junit4] 2> "_enqueue_time_":1857481637015684,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}"]}} with context properties: {BEFORE_ACTION=[compute]}
[junit4] 2> 522095 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSearchRate
[junit4] 2> 522095 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 522096 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 522096 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 522096 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 522096 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 100
[junit4] 2> 522096 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 80
[junit4] 2> 522096 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 522122 INFO (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10072_solr&type=NRT
[junit4] 2> 522130 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [testSearchRate]
[junit4] 2> 522130 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 1
[junit4] 2> 522130 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 522130 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 20
[junit4] 2> 522130 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * testSearchRate 20
[junit4] 2> 522131 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - active 20
[junit4] 2> 522131 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 522131 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 522131 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 522131 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 1
[junit4] 2> 522131 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 522132 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: search_rate_trigger
[junit4] 2> 522132 INFO (TEST-TestLargeCluster.testSearchRate-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLargeCluster -Dtests.method=testSearchRate -Dtests.seed=B8A6057108377028 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=fr -Dtests.timezone=America/Pangnirtung -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 1.34s J1 | TestLargeCluster.testSearchRate <<<
[junit4] > Throwable #1: java.lang.AssertionError: The trigger did not fire at all
[junit4] > at __randomizedtesting.SeedInfo.seed([B8A6057108377028:E5EE1BF8C7F1D667]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 522158 INFO (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10056_solr&type=NRT
[junit4] 2> 522267 INFO (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10059_solr&type=NRT
[junit4] 2> 522268 INFO (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 1857499057 sessionWrapper.createTime 1857488109486784, this.sessionWrapper.createTime 1857488109486784
[junit4] 2> 522268 DEBUG (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"6995c50337ddeT62ebd1akbzicnr4uc86fnbf90",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1857471621856734,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10091_solr":4.0,
[junit4] 2> "127.0.0.1:10016_solr":4.0,
[junit4] 2> "127.0.0.1:10086_solr":4.0,
[junit4] 2> "127.0.0.1:10089_solr":4.0,
[junit4] 2> "127.0.0.1:10020_solr":4.0,
[junit4] 2> "127.0.0.1:10065_solr":4.0,
[junit4] 2> "127.0.0.1:10003_solr":4.0,
[junit4] 2> "127.0.0.1:10088_solr":4.0,
[junit4] 2> "127.0.0.1:10027_solr":4.0,
[junit4] 2> "127.0.0.1:10007_solr":4.0},
[junit4] 2> "__start__":1,
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node10\":{\n \"core\":\"testSearchRate_shard1_replica_n10\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10020_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSearchRate_shard1_replica_n3\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10065_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSearchRate_shard1_replica_n1\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10003_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node8\":{\n \"core\":\"testSearchRate_shard1_replica_n8\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node7\":{\n \"core\":\"testSearchRate_shard1_replica_n7\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10016_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node4\":{\n \"core\":\"testSearchRate_shard1_replica_n4\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10086_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node5\":{\n \"core\":\"testSearchRate_shard1_replica_n5\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10089_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node9\":{\n \"core\":\"testSearchRate_shard1_replica_n9\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10088_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node6\":{\n \"core\":\"testSearchRate_shard1_replica_n6\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10027_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"testSearchRate_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10091_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}"],
[junit4] 2> "collection":{"testSearchRate":40.0},
[junit4] 2> "shard":{"testSearchRate":{"shard1":40.0}},
[junit4] 2> "_enqueue_time_":1857481637015684,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@3d4a9e16, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@3945e12c, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@4f311059], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
[junit4] 2> 522269 DEBUG (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10072_solr&type=NRT
[junit4] 2> 522271 INFO (simCloudManagerPool-974-thread-25) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10072_solr for creating new replica
[junit4] 2> 522487 DEBUG (simCloudManagerPool-974-thread-26) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testSearchRate / shard2: {"core_node18":{
[junit4] 2> "node_name":"127.0.0.1:10069_solr",
[junit4] 2> "core":"testSearchRate_shard2_replica_n18",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testSearchRate"}}
[junit4] 2> 522515 INFO (TEST-TestLargeCluster.testAddNode-seed#[B8A6057108377028]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAddNode
[junit4] 2> 522515 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[B8A6057108377028]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 522515 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 522515 INFO (TEST-TestLargeCluster.testAddNode-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 522516 WARN (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Unexpected exception while trying to delete znode: /autoscaling/triggerState/search_rate_trigger/execute/op-0000000000
[junit4] 2> java.util.NoSuchElementException: /autoscaling/triggerState/search_rate_trigger/execute/op-0000000000
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimDistribStateManager.removeData(SimDistribStateManager.java:477)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:91)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:300)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 522516 DEBUG (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=testSearchRate&shard=shard1&node=127.0.0.1:10056_solr&type=NRT
[junit4] 2> 522517 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 522517 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 522517 DEBUG (TEST-TestLargeCluster.testAddNode-seed#[B8A6057108377028]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 522517 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:154)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 522517 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 522518 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 522518 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 522518 WARN (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> java.lang.Exception: Error executing action: execute for trigger event: {
[junit4] 2> "id":"6995c50337ddeT62ebd1akbzicnr4uc86fnbf90",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1857471621856734,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10091_solr":4.0,
[junit4] 2> "127.0.0.1:10016_solr":4.0,
[junit4] 2> "127.0.0.1:10086_solr":4.0,
[junit4] 2> "127.0.0.1:10089_solr":4.0,
[junit4] 2> "127.0.0.1:10020_solr":4.0,
[junit4] 2> "127.0.0.1:10065_solr":4.0,
[junit4] 2> "127.0.0.1:10003_solr":4.0,
[junit4] 2> "127.0.0.1:10088_solr":4.0,
[junit4] 2> "127.0.0.1:10027_solr":4.0,
[junit4] 2> "127.0.0.1:10007_solr":4.0},
[junit4] 2> "__start__":1,
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node10\":{\n \"core\":\"testSearchRate_shard1_replica_n10\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10020_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSearchRate_shard1_replica_n3\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10065_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSearchRate_shard1_replica_n1\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10003_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node8\":{\n \"core\":\"testSearchRate_shard1_replica_n8\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node7\":{\n \"core\":\"testSearchRate_shard1_replica_n7\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10016_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node4\":{\n \"core\":\"testSearchRate_shard1_replica_n4\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10086_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node5\":{\n \"core\":\"testSearchRate_shard1_replica_n5\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10089_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node9\":{\n \"core\":\"testSearchRate_shard1_replica_n9\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10088_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node6\":{\n \"core\":\"testSearchRate_shard1_replica_n6\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10027_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"testSearchRate_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10091_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}"],
[junit4] 2> "collection":{"testSearchRate":40.0},
[junit4] 2> "shard":{"testSearchRate":{"shard1":40.0}},
[junit4] 2> "_enqueue_time_":1857481637015684,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:303)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unexpected exception while processing event: {
[junit4] 2> "id":"6995c50337ddeT62ebd1akbzicnr4uc86fnbf90",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1857471621856734,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10091_solr":4.0,
[junit4] 2> "127.0.0.1:10016_solr":4.0,
[junit4] 2> "127.0.0.1:10086_solr":4.0,
[junit4] 2> "127.0.0.1:10089_solr":4.0,
[junit4] 2> "127.0.0.1:10020_solr":4.0,
[junit4] 2> "127.0.0.1:10065_solr":4.0,
[junit4] 2> "127.0.0.1:10003_solr":4.0,
[junit4] 2> "127.0.0.1:10088_solr":4.0,
[junit4] 2> "127.0.0.1:10027_solr":4.0,
[junit4] 2> "127.0.0.1:10007_solr":4.0},
[junit4] 2> "__start__":1,
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node10\":{\n \"core\":\"testSearchRate_shard1_replica_n10\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10020_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node3\":{\n \"core\":\"testSearchRate_shard1_replica_n3\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10065_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"testSearchRate_shard1_replica_n1\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10003_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node8\":{\n \"core\":\"testSearchRate_shard1_replica_n8\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10007_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node7\":{\n \"core\":\"testSearchRate_shard1_replica_n7\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10016_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node4\":{\n \"core\":\"testSearchRate_shard1_replica_n4\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10086_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node5\":{\n \"core\":\"testSearchRate_shard1_replica_n5\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10089_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node9\":{\n \"core\":\"testSearchRate_shard1_replica_n9\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10088_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node6\":{\n \"core\":\"testSearchRate_shard1_replica_n6\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10027_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"testSearchRate_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":4.0,\n \"node_name\":\"127.0.0.1:10091_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":4.0,\n \"shard\":\"shard1\",\n \"collection\":\"testSearchRate\"}}"],
[junit4] 2> "collection":{"testSearchRate":40.0},
[junit4] 2> "shard":{"testSearchRate":{"shard1":40.0}},
[junit4] 2> "_enqueue_time_":1857481637015684,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"testSearchRate\",\n \"second\":\"shard1\"}}}"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:120)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:300)
[junit4] 2> ... 6 more
[junit4] 2> Caused by: org.apache.solr.common.SolrException: Unexpected exception executing operation: action=ADDREPLICA&async=search_rate_trigger/6995c50337ddeT62ebd1akbzicnr4uc86fnbf90/0&waitForFinalState=true&collection=testSearchRate&shard=shard1&node=127.0.0.1:10056_solr&type=NRT
[junit4] 2> at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:110)
[junit4] 2> ... 7 more
[junit4] 2> Caused by: java.io.IOException: java.lang.InterruptedException
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:462)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ExecutePlanAction.process(ExecutePlanAction.java:80)
[junit4] 2> ... 7 more
[junit4] 2> Caused by: java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404)
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:460)
[junit4] 2> ... 8 more
[junit4] 2> 522520 DEBUG (AutoscalingActionExecutor-976-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 28898 ms for event id=6995c50337ddeT62ebd1akbzicnr4uc86fnbf90
[junit4] 2> 522518 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
[junit4] 2> 522535 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 522559 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 522559 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 522559 DEBUG (ScheduledTrigger-978-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 522595 DEBUG (ScheduledTrigger-978-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 522742 DEBUG (simCloudManagerPool-977-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 522742 DEBUG (simCloudManagerPool-977-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 522782 DEBUG (ScheduledTrigger-978-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 522891 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 522895 DEBUG (simCloudManagerPool-977-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 522897 DEBUG (simCloudManagerPool-977-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 522897 DEBUG (simCloudManagerPool-977-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=compute, class=org.apache.solr.cloud.autoscaling.ComputePlanAction}, {name=execute, class=org.apache.solr.cloud.autoscaling.ExecutePlanAction}], enabled=true}
[junit4] 2> 522898 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 522898 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 522915 DEBUG (ScheduledTrigger-978-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 522915 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 522920 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 522935 DEBUG (ScheduledTrigger-978-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 522964 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523000 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523021 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523175 DEBUG (ScheduledTrigger-978-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523175 DEBUG (ScheduledTrigger-978-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523175 DEBUG (ScheduledTrigger-978-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523276 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523276 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523276 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523312 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523312 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523312 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523349 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523349 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523349 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523385 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523385 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523385 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523414 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523414 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523414 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523450 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523450 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523450 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523470 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523470 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523470 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523519 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523519 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523519 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523539 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523539 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523539 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523559 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523560 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523560 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523584 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523585 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523585 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523632 DEBUG (ScheduledTrigger-978-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523632 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523632 DEBUG (ScheduledTrigger-978-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523665 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523665 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523665 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523701 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523701 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523701 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523721 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523721 DEBUG (ScheduledTrigger-978-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523722 DEBUG (ScheduledTrigger-978-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523742 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523742 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523742 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523762 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523762 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523762 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523798 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523799 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523799 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523839 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523855 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523855 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523892 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523892 DEBUG (ScheduledTrigger-978-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523892 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523916 DEBUG (ScheduledTrigger-978-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523916 DEBUG (ScheduledTrigger-978-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523916 DEBUG (ScheduledTrigger-978-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523936 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523936 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523936 DEBUG (ScheduledTrigger-978-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523959 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523960 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523960 DEBUG (ScheduledTrigger-978-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 523980 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 523980 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 523980 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 524000 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 524000 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 524000 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 524020 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 524020 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 524020 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 524050 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 524051 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 524051 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 524090 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 524090 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[junit4] 2> 524090 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 524111 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 524111 DEBUG (ScheduledTrigger-978-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 100
[...truncated too long message...]
s.SimSolrCloudTestCase #############################################
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 80
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 0
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 20
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10004_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10017_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10020_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10021_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10033_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10035_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10036_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10047_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10049_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10051_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10081_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10084_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10095_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10096_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10102_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10104_solr
[junit4] 2> 575169 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10118_solr
[junit4] 2> 575170 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10119_solr
[junit4] 2> 575170 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10120_solr
[junit4] 2> 575170 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10121_solr
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [testNodeLost]
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 3
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 2
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 200
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * testNodeLost 200
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - active 200
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - DELETESTATUS 40
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - MOVEREPLICA 41
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - REQUESTSTATUS 40
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 1
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 18
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: .auto_add_replicas
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 4
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 4
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_trigger
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 5
[junit4] 2> 575183 INFO (TEST-TestLargeCluster.testNodeLost-seed#[B8A6057108377028]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 5
[junit4] 2> 575191 DEBUG (ScheduledTrigger-984-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 80
[junit4] 2> 575191 DEBUG (ScheduledTrigger-984-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 80
[junit4] 2> 575195 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 575267 DEBUG (ScheduledTrigger-984-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 80
[junit4] 2> 575268 DEBUG (ScheduledTrigger-984-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 80
[junit4] 2> 575279 DEBUG (SUITE-TestLargeCluster-seed#[B8A6057108377028]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestLargeCluster_B8A6057108377028-001
[junit4] 2> Jan 31, 2018 12:52:21 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 4 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=32, maxMBSortInHeap=7.480542699887797, sim=RandomSimilarity(queryNorm=true): {}, locale=fr, timezone=America/Pangnirtung
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=283926840,total=425721856
[junit4] 2> NOTE: All tests run in this JVM: [HdfsRestartWhileUpdatingTest, LoggingHandlerTest, TestSweetSpotSimilarityFactory, DirectUpdateHandlerOptimizeTest, TestNumericRangeQuery64, TestInitQParser, TestStandardQParsers, TestFastLRUCache, FileUtilsTest, PrimitiveFieldTypeTest, MultiThreadedOCPTest, SortByFunctionTest, TestOrdValues, DeleteStatusTest, DistributedFacetPivotSmallTest, TestReRankQParserPlugin, DistribJoinFromCollectionTest, TestDistribStateManager, CdcrVersionReplicationTest, HighlighterMaxOffsetTest, BasicFunctionalityTest, TestSolrCloudWithSecureImpersonation, PathHierarchyTokenizerFactoryTest, RestartWhileUpdatingTest, SmileWriterTest, TestBulkSchemaConcurrent, SystemInfoHandlerTest, SolrGangliaReporterTest, HighlighterTest, InfixSuggestersTest, GraphQueryTest, TestPartialUpdateDeduplication, TestPivotHelperCode, IndexSchemaTest, TestLargeCluster]
[junit4] Completed [58/773 (1!)] on J1 in 55.67s, 4 tests, 1 failure <<< FAILURES!
[...truncated 50610 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 343 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/343/
4 tests failed.
FAILED: org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal
Error Message:
concurrent alias creation failed java.lang.IllegalStateException: Connection pool shut down
Stack Trace:
java.lang.AssertionError: concurrent alias creation failed java.lang.IllegalStateException: Connection pool shut down
at __randomizedtesting.SeedInfo.seed([963C9F53888D1D35:52A793012592AA0F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal(ConcurrentCreateRoutedAliasTest.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore.test
Error Message:
expected:<COMPLETED> but was:<RUNNING>
Stack Trace:
java.lang.AssertionError: expected:<COMPLETED> but was:<RUNNING>
at __randomizedtesting.SeedInfo.seed([963C9F53888D1D35:1E68A089267170CD]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.testBackupAndRestore(AbstractCloudBackupRestoreTestCase.java:289)
at org.apache.solr.cloud.api.collections.AbstractCloudBackupRestoreTestCase.test(AbstractCloudBackupRestoreTestCase.java:142)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([963C9F53888D1D35:2C30A8DCD765CB7A]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.junit.Assert.assertNull(Assert.java:562)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([963C9F53888D1D35:CB7481DA474BBB7A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11947 lines...]
[junit4] Suite: org.apache.solr.cloud.api.collections.TestHdfsCloudBackupRestore
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/init-core-data-001
[junit4] 2> 917264 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 917264 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 917301 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 917302 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 919775 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 919993 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 920000 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 920156 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_51411_hdfs____vvszqa/webapp
[junit4] 2> 926743 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:51411
[junit4] 2> 929257 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 929259 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 929359 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_47051_datanode____.ll8ncg/webapp
[junit4] 2> 935682 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:47051
[junit4] 2> 936844 WARN (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 936850 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 937262 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_38677_datanode____.josk94/webapp
[junit4] 2> 938918 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-001/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-001/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:32939) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 938995 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x3e857f447dce87: from storage DS-85f543fc-ae18-4654-8ea7-457d58eb022b node DatanodeRegistration(127.0.0.1:56062, datanodeUuid=3efa0368-b7ff-4cc5-9a36-8589c6421d1d, infoPort=59133, infoSecurePort=0, ipcPort=40369, storageInfo=lv=-56;cid=testClusterID;nsid=1396815523;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 938995 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x3e857f447dce87: from storage DS-d2ca3cce-9ac1-4d21-af71-0b436165d624 node DatanodeRegistration(127.0.0.1:56062, datanodeUuid=3efa0368-b7ff-4cc5-9a36-8589c6421d1d, infoPort=59133, infoSecurePort=0, ipcPort=40369, storageInfo=lv=-56;cid=testClusterID;nsid=1396815523;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 941957 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:38677
[junit4] 2> 947319 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-001/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-001/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:32939) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 947477 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x3e85813cbddc44: from storage DS-f018afea-3b42-41f5-8833-bcfa764f588a node DatanodeRegistration(127.0.0.1:45986, datanodeUuid=e7c0d423-4302-4585-805e-0a6cf4aef7dc, infoPort=48285, infoSecurePort=0, ipcPort=42023, storageInfo=lv=-56;cid=testClusterID;nsid=1396815523;c=0), blocks: 0, hasStaleStorage: true, processing time: 16 msecs
[junit4] 2> 947477 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x3e85813cbddc44: from storage DS-5e3846f9-fd92-46cd-81a4-2b9391328869 node DatanodeRegistration(127.0.0.1:45986, datanodeUuid=e7c0d423-4302-4585-805e-0a6cf4aef7dc, infoPort=48285, infoSecurePort=0, ipcPort=42023, storageInfo=lv=-56;cid=testClusterID;nsid=1396815523;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 949656 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002
[junit4] 2> 949656 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 949657 INFO (Thread-875) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 949657 INFO (Thread-875) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 949674 ERROR (Thread-875) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 949758 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46976
[junit4] 2> 950061 INFO (zkConnectionManagerCallback-158-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 950134 INFO (jetty-launcher-155-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 950146 INFO (jetty-launcher-155-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 950178 INFO (jetty-launcher-155-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 950178 INFO (jetty-launcher-155-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 950178 INFO (jetty-launcher-155-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 950179 INFO (jetty-launcher-155-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f6699ad{/solr,null,AVAILABLE}
[junit4] 2> 950179 INFO (jetty-launcher-155-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@d12c520{HTTP/1.1,[http/1.1]}{127.0.0.1:56822}
[junit4] 2> 950179 INFO (jetty-launcher-155-thread-2) [ ] o.e.j.s.Server Started @984505ms
[junit4] 2> 950179 INFO (jetty-launcher-155-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56822}
[junit4] 2> 950180 ERROR (jetty-launcher-155-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 950180 INFO (jetty-launcher-155-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 950180 INFO (jetty-launcher-155-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 950180 INFO (jetty-launcher-155-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 950180 INFO (jetty-launcher-155-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-30T21:05:27.580Z
[junit4] 2> 950182 INFO (jetty-launcher-155-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 950182 INFO (jetty-launcher-155-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 950182 INFO (jetty-launcher-155-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 950254 INFO (jetty-launcher-155-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@75df0b21{/solr,null,AVAILABLE}
[junit4] 2> 950256 INFO (jetty-launcher-155-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@49915d65{HTTP/1.1,[http/1.1]}{127.0.0.1:45076}
[junit4] 2> 950256 INFO (jetty-launcher-155-thread-1) [ ] o.e.j.s.Server Started @984582ms
[junit4] 2> 950256 INFO (jetty-launcher-155-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45076}
[junit4] 2> 950257 ERROR (jetty-launcher-155-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 950292 INFO (jetty-launcher-155-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 950292 INFO (jetty-launcher-155-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 950292 INFO (jetty-launcher-155-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 950292 INFO (jetty-launcher-155-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-30T21:05:27.692Z
[junit4] 2> 950736 INFO (zkConnectionManagerCallback-162-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 950910 INFO (jetty-launcher-155-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 950944 INFO (jetty-launcher-155-thread-1) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 950962 INFO (jetty-launcher-155-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46976/solr
[junit4] 2> 950963 INFO (zkConnectionManagerCallback-160-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 951022 INFO (jetty-launcher-155-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 951036 INFO (jetty-launcher-155-thread-2) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 951110 INFO (jetty-launcher-155-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46976/solr
[junit4] 2> 951181 INFO (zkConnectionManagerCallback-168-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 951216 INFO (zkConnectionManagerCallback-170-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 951514 INFO (zkConnectionManagerCallback-172-thread-1-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 951670 INFO (zkConnectionManagerCallback-174-thread-1-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 956631 INFO (jetty-launcher-155-thread-2) [n:127.0.0.1:56822_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 956633 INFO (jetty-launcher-155-thread-2) [n:127.0.0.1:56822_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56822_solr
[junit4] 2> 956687 INFO (jetty-launcher-155-thread-2) [n:127.0.0.1:56822_solr ] o.a.s.c.Overseer Overseer (id=73210912353026054-127.0.0.1:56822_solr-n_0000000000) starting
[junit4] 2> 956985 INFO (jetty-launcher-155-thread-2) [n:127.0.0.1:56822_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56822_solr
[junit4] 2> 957069 INFO (zkCallback-173-thread-1-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 957633 INFO (jetty-launcher-155-thread-2) [n:127.0.0.1:56822_solr ] o.a.s.c.b.r.BackupRepositoryFactory Added backup repository with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://localhost:32939/solr,solr.hdfs.confdir=}}
[junit4] 2> 957633 INFO (jetty-launcher-155-thread-2) [n:127.0.0.1:56822_solr ] o.a.s.c.b.r.BackupRepositoryFactory Default configuration for backup repository is with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://localhost:32939/solr,solr.hdfs.confdir=}}
[junit4] 2> 958444 INFO (jetty-launcher-155-thread-2) [n:127.0.0.1:56822_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 958710 INFO (jetty-launcher-155-thread-1) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 958720 INFO (jetty-launcher-155-thread-1) [n:127.0.0.1:45076_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 958728 INFO (jetty-launcher-155-thread-2) [n:127.0.0.1:56822_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 958728 INFO (jetty-launcher-155-thread-2) [n:127.0.0.1:56822_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 958741 INFO (jetty-launcher-155-thread-1) [n:127.0.0.1:45076_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45076_solr
[junit4] 2> 958742 INFO (zkCallback-173-thread-1-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 958808 INFO (jetty-launcher-155-thread-2) [n:127.0.0.1:56822_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node2/.
[junit4] 2> 959161 INFO (zkCallback-171-thread-1-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 960062 INFO (jetty-launcher-155-thread-1) [n:127.0.0.1:45076_solr ] o.a.s.c.b.r.BackupRepositoryFactory Added backup repository with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://localhost:32939/solr,solr.hdfs.confdir=}}
[junit4] 2> 960065 INFO (jetty-launcher-155-thread-1) [n:127.0.0.1:45076_solr ] o.a.s.c.b.r.BackupRepositoryFactory Default configuration for backup repository is with configuration params {type = repository,name = hdfs,class = org.apache.solr.core.backup.repository.HdfsBackupRepository,attributes = {name=hdfs, class=org.apache.solr.core.backup.repository.HdfsBackupRepository},args = {location=/backup,solr.hdfs.home=hdfs://localhost:32939/solr,solr.hdfs.confdir=}}
[junit4] 2> 960734 INFO (jetty-launcher-155-thread-1) [n:127.0.0.1:45076_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 961038 INFO (jetty-launcher-155-thread-1) [n:127.0.0.1:45076_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 961038 INFO (jetty-launcher-155-thread-1) [n:127.0.0.1:45076_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 961069 INFO (jetty-launcher-155-thread-1) [n:127.0.0.1:45076_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node1/.
[junit4] 2> 961771 INFO (zkConnectionManagerCallback-180-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 961940 INFO (zkConnectionManagerCallback-184-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 961946 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 961949 INFO (SUITE-TestHdfsCloudBackupRestore-seed#[963C9F53888D1D35]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46976/solr ready
[junit4] 2> 962521 INFO (TEST-TestHdfsCloudBackupRestore.test-seed#[963C9F53888D1D35]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 962583 INFO (qtp527904507-1865) [n:127.0.0.1:56822_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&router.name=implicit&version=2&pullReplicas=1&shards=shard1,shard2&property.customKey=customValue&maxShardsPerNode=3&router.field=shard_s&autoAddReplicas=true&name=hdfsbackuprestore&nrtReplicas=2&action=CREATE&tlogReplicas=0&wt=javabin and sendToOCPQueue=true
[junit4] 2> 962618 INFO (OverseerThreadFactory-775-thread-1-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection hdfsbackuprestore
[junit4] 2> 962619 WARN (OverseerThreadFactory-775-thread-1-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 3 on collection hdfsbackuprestore is higher than the number of Solr instances currently live or live and part of your createNodeSet(2). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 962993 INFO (OverseerStateUpdate-73210912353026054-127.0.0.1:56822_solr-n_0000000000) [n:127.0.0.1:56822_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"hdfsbackuprestore_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:45076/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 963025 INFO (OverseerStateUpdate-73210912353026054-127.0.0.1:56822_solr-n_0000000000) [n:127.0.0.1:56822_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"hdfsbackuprestore_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:56822/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 963059 INFO (OverseerStateUpdate-73210912353026054-127.0.0.1:56822_solr-n_0000000000) [n:127.0.0.1:56822_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"hdfsbackuprestore_shard1_replica_p4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:45076/solr",
[junit4] 2> "type":"PULL",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 963125 INFO (OverseerStateUpdate-73210912353026054-127.0.0.1:56822_solr-n_0000000000) [n:127.0.0.1:56822_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"hdfsbackuprestore_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:56822/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 963193 INFO (OverseerStateUpdate-73210912353026054-127.0.0.1:56822_solr-n_0000000000) [n:127.0.0.1:56822_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"hdfsbackuprestore_shard2_replica_n8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:45076/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 963329 INFO (OverseerStateUpdate-73210912353026054-127.0.0.1:56822_solr-n_0000000000) [n:127.0.0.1:56822_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"hdfsbackuprestore",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"hdfsbackuprestore_shard2_replica_p10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:56822/solr",
[junit4] 2> "type":"PULL",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 963722 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node3&name=hdfsbackuprestore_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin
[junit4] 2> 963722 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 963770 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=PULL&property.customKey=customValue&coreNodeName=core_node12&name=hdfsbackuprestore_shard2_replica_p10&action=CREATE&numShards=2&shard=shard2&wt=javabin
[junit4] 2> 963777 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 964183 INFO (zkCallback-173-thread-1-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 964203 INFO (zkCallback-173-thread-2-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 964204 INFO (zkCallback-171-thread-1-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 964330 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node11&name=hdfsbackuprestore_shard2_replica_n8&action=CREATE&numShards=2&shard=shard2&wt=javabin
[junit4] 2> 964335 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node5&name=hdfsbackuprestore_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&wt=javabin
[junit4] 2> 964357 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=PULL&property.customKey=customValue&coreNodeName=core_node7&name=hdfsbackuprestore_shard1_replica_p4&action=CREATE&numShards=2&shard=shard1&wt=javabin
[junit4] 2> 964398 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node9&name=hdfsbackuprestore_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin
[junit4] 2> 964504 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 964538 INFO (zkCallback-173-thread-3-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 964554 INFO (zkCallback-173-thread-2-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 964554 INFO (zkCallback-171-thread-1-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 964587 INFO (zkCallback-171-thread-2-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 964686 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.s.IndexSchema [hdfsbackuprestore_shard1_replica_n1] Schema name=minimal
[junit4] 2> 964823 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 964823 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard1_replica_n1' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 964824 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard1.replica_n1' (registry 'solr.core.hdfsbackuprestore.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 964824 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 964824 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SolrCore [[hdfsbackuprestore_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node1/hdfsbackuprestore_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node1/./hdfsbackuprestore_shard1_replica_n1/data/]
[junit4] 2> 965288 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 965450 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.s.IndexSchema [hdfsbackuprestore_shard2_replica_p10] Schema name=minimal
[junit4] 2> 965497 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 965497 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard2_replica_p10' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 965498 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard2.replica_p10' (registry 'solr.core.hdfsbackuprestore.shard2.replica_p10') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 965499 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 965499 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.c.SolrCore [[hdfsbackuprestore_shard2_replica_p10] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node2/hdfsbackuprestore_shard2_replica_p10], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node2/./hdfsbackuprestore_shard2_replica_p10/data/]
[junit4] 2> 966125 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 966163 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 966168 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 966212 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 966727 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.s.IndexSchema [hdfsbackuprestore_shard2_replica_n6] Schema name=minimal
[junit4] 2> 966790 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 966790 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard2_replica_n6' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 966791 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard2.replica_n6' (registry 'solr.core.hdfsbackuprestore.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 966791 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 966792 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.SolrCore [[hdfsbackuprestore_shard2_replica_n6] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node2/hdfsbackuprestore_shard2_replica_n6], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node2/./hdfsbackuprestore_shard2_replica_n6/data/]
[junit4] 2> 966990 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.s.IndexSchema [hdfsbackuprestore_shard1_replica_p4] Schema name=minimal
[junit4] 2> 967014 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.s.IndexSchema [hdfsbackuprestore_shard1_replica_n2] Schema name=minimal
[junit4] 2> 967085 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 967085 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard1_replica_p4' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 967086 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 967086 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard1_replica_n2' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 967097 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard1.replica_p4' (registry 'solr.core.hdfsbackuprestore.shard1.replica_p4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 967097 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 967097 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.c.SolrCore [[hdfsbackuprestore_shard1_replica_p4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node1/hdfsbackuprestore_shard1_replica_p4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node1/./hdfsbackuprestore_shard1_replica_p4/data/]
[junit4] 2> 967269 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard1.replica_n2' (registry 'solr.core.hdfsbackuprestore.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 967269 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 967269 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.SolrCore [[hdfsbackuprestore_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node2/hdfsbackuprestore_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node2/./hdfsbackuprestore_shard1_replica_n2/data/]
[junit4] 2> 967777 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.s.IndexSchema [hdfsbackuprestore_shard2_replica_n8] Schema name=minimal
[junit4] 2> 968010 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 968010 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.CoreContainer Creating SolrCore 'hdfsbackuprestore_shard2_replica_n8' using configuration from collection hdfsbackuprestore, trusted=true
[junit4] 2> 968011 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.hdfsbackuprestore.shard2.replica_n8' (registry 'solr.core.hdfsbackuprestore.shard2.replica_n8') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@52a4746c
[junit4] 2> 968011 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 968011 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SolrCore [[hdfsbackuprestore_shard2_replica_n8] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node1/hdfsbackuprestore_shard2_replica_n8], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestHdfsCloudBackupRestore_963C9F53888D1D35-001/tempDir-002/node1/./hdfsbackuprestore_shard2_replica_n8/data/]
[junit4] 2> 969016 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 969016 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 969019 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 969019 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 969045 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 969045 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 969047 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@ab1621a[hdfsbackuprestore_shard1_replica_n1] main]
[junit4] 2> 969071 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 969081 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 969158 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 969161 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.s.SolrIndexSearcher Opening [Searcher@6419745b[hdfsbackuprestore_shard2_replica_p10] main]
[junit4] 2> 969259 INFO (searcherExecutor-780-thread-1-processing-n:127.0.0.1:45076_solr x:hdfsbackuprestore_shard1_replica_n1 s:shard1 c:hdfsbackuprestore r:core_node3) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SolrCore [hdfsbackuprestore_shard1_replica_n1] Registered new searcher Searcher@ab1621a[hdfsbackuprestore_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 969318 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1591052962856173568
[junit4] 2> 969341 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 969342 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 969343 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 970013 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.c.ZkController hdfsbackuprestore_shard2_replica_p10 starting background replication from leader
[junit4] 2> 970015 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 970079 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 970079 INFO (qtp527904507-1861) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 970081 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 970129 INFO (searcherExecutor-781-thread-1-processing-n:127.0.0.1:56822_solr x:hdfsbackuprestore_shard2_replica_p10 s:shard2 c:hdfsbackuprestore r:core_node12) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.c.SolrCore [hdfsbackuprestore_shard2_replica_p10] Registered new searcher Searcher@6419745b[hdfsbackuprestore_shard2_replica_p10] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 970145 INFO (zkCallback-173-thread-2-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 970145 INFO (zkCallback-173-thread-3-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 970149 INFO (zkCallback-171-thread-2-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 970149 INFO (zkCallback-171-thread-1-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 970193 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 970193 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 970617 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 970617 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 970619 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 970619 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 970657 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@23ae362d[hdfsbackuprestore_shard2_replica_n6] main]
[junit4] 2> 970659 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 970660 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 970673 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 970673 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1591052964276994048
[junit4] 2> 970677 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.s.SolrIndexSearcher Opening [Searcher@7f2f4e9c[hdfsbackuprestore_shard1_replica_p4] main]
[junit4] 2> 970695 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 970695 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 970697 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 970713 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 970714 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 970716 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 970716 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 970845 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.s.SolrIndexSearcher Opening [Searcher@1414bf[hdfsbackuprestore_shard2_replica_n8] main]
[junit4] 2> 970846 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 970846 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 970862 INFO (searcherExecutor-782-thread-1-processing-n:127.0.0.1:56822_solr x:hdfsbackuprestore_shard2_replica_n6 s:shard2 c:hdfsbackuprestore r:core_node9) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.SolrCore [hdfsbackuprestore_shard2_replica_n6] Registered new searcher Searcher@23ae362d[hdfsbackuprestore_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 970863 INFO (searcherExecutor-783-thread-1-processing-n:127.0.0.1:45076_solr x:hdfsbackuprestore_shard1_replica_p4 s:shard1 c:hdfsbackuprestore r:core_node7) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.c.SolrCore [hdfsbackuprestore_shard1_replica_p4] Registered new searcher Searcher@7f2f4e9c[hdfsbackuprestore_shard1_replica_p4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 971467 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 971467 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 971468 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 971469 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 971470 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 971470 INFO (qtp1066405676-1870) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1591052965112709120
[junit4] 2> 971489 INFO (searcherExecutor-787-thread-1-processing-n:127.0.0.1:45076_solr x:hdfsbackuprestore_shard2_replica_n8 s:shard2 c:hdfsbackuprestore r:core_node11) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.SolrCore [hdfsbackuprestore_shard2_replica_n8] Registered new searcher Searcher@1414bf[hdfsbackuprestore_shard2_replica_n8] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 971620 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@554bd268[hdfsbackuprestore_shard1_replica_n2] main]
[junit4] 2> 971621 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.c.ZkController hdfsbackuprestore_shard1_replica_p4 starting background replication from leader
[junit4] 2> 971622 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 971635 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 971635 INFO (qtp1066405676-1877) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node7 x:hdfsbackuprestore_shard1_replica_p4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 971665 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 971673 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 971674 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 971675 INFO (qtp527904507-1863) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1591052965327667200
[junit4] 2> 971786 INFO (searcherExecutor-784-thread-1-processing-n:127.0.0.1:56822_solr x:hdfsbackuprestore_shard1_replica_n2 s:shard1 c:hdfsbackuprestore r:core_node5) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.SolrCore [hdfsbackuprestore_shard1_replica_n2] Registered new searcher Searcher@554bd268[hdfsbackuprestore_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 971904 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 971905 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 971905 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:56822/solr/hdfsbackuprestore_shard2_replica_n6/
[junit4] 2> 971932 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=hdfsbackuprestore_shard2_replica_n6 url=http://127.0.0.1:56822/solr START replicas=[http://127.0.0.1:45076/solr/hdfsbackuprestore_shard2_replica_n8/] nUpdates=100
[junit4] 2> 971978 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 971978 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 971978 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:45076/solr/hdfsbackuprestore_shard1_replica_n1/
[junit4] 2> 971979 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=hdfsbackuprestore_shard1_replica_n1 url=http://127.0.0.1:45076/solr START replicas=[http://127.0.0.1:56822/solr/hdfsbackuprestore_shard1_replica_n2/] nUpdates=100
[junit4] 2> 972002 INFO (zkCallback-171-thread-1-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 972003 INFO (zkCallback-173-thread-3-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 972003 INFO (zkCallback-173-thread-2-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 972073 INFO (zkCallback-171-thread-3-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 972149 INFO (qtp527904507-1938) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard1 r:core_node5 x:hdfsbackuprestore_shard1_replica_n2] o.a.s.c.S.Request [hdfsbackuprestore_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 972298 INFO (qtp1066405676-1872) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard2 r:core_node11 x:hdfsbackuprestore_shard2_replica_n8] o.a.s.c.S.Request [hdfsbackuprestore_shard2_replica_n8] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=124
[junit4] 2> 972301 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.u.PeerSync PeerSync: core=hdfsbackuprestore_shard2_replica_n6 url=http://127.0.0.1:56822/solr DONE. We have no versions. sync failed.
[junit4] 2> 972325 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 972325 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 972325 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 972373 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=hdfsbackuprestore_shard1_replica_n1 url=http://127.0.0.1:45076/solr DONE. We have no versions. sync failed.
[junit4] 2> 972373 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 972373 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 972373 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 973709 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:45076/solr/hdfsbackuprestore_shard1_replica_n1/ shard1
[junit4] 2> 973763 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:56822/solr/hdfsbackuprestore_shard2_replica_n6/ shard2
[junit4] 2> 973892 INFO (zkCallback-173-thread-1-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 973893 INFO (zkCallback-173-thread-3-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 973893 INFO (zkCallback-171-thread-3-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 973893 INFO (zkCallback-171-thread-1-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 973950 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 973951 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 973951 INFO (indexFetcher-797-thread-1) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node12 x:hdfsbackuprestore_shard2_replica_p10] o.a.s.h.IndexFetcher Replica core_node9 is leader but it's state is down, skipping replication
[junit4] 2> 973961 INFO (qtp1066405676-1871) [n:127.0.0.1:45076_solr c:hdfsbackuprestore s:shard1 r:core_node3 x:hdfsbackuprestore_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node3&name=hdfsbackuprestore_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&wt=javabin} status=0 QTime=10238
[junit4] 2> 973963 INFO (qtp527904507-1867) [n:127.0.0.1:56822_solr c:hdfsbackuprestore s:shard2 r:core_node9 x:hdfsbackuprestore_shard2_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&collection=hdfsbackuprestore&version=2&replicaType=NRT&property.customKey=customValue&coreNodeName=core_node9&name=hdfsbackuprestore_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&wt=javabin} status=0 QTime=9565
[junit4] 2> 974070 INFO (zkCallback-171-thread-3-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 974071 INFO (zkCallback-171-thread-1-processing-n:127.0.0.1:45076_solr) [n:127.0.0.1:45076_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating... (live nodes size: [2])
[junit4] 2> 974074 INFO (zkCallback-173-thread-1-processing-n:127.0.0.1:56822_solr) [n:127.0.0.1:56822_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/hdfsbackuprestore/state.json] for collection [hdfsbackuprestore] has occurred - updating...
[...truncated too long message...]
ve/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/bcprov-jdk15on-1.54.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/commons-math3-3.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/analysis/icu/lucene-analyzers-icu-7.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/contrib/analysis-extras/lib/icu4j-60.2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180130_204902_6387734699034853613405.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180130_204902_638738875226015830346.suites -stdin
[junit4] ERROR: JVM J2 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180130_204902_6384939566263072388628.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180130_204902_6381482280639190651496.syserr
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
[junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] at java.lang.Thread.run(Thread.java:748)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:836: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:780: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build.xml:262: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/common-build.xml:556: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:1513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:1038: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180130_204902_6397246301557099666982.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180130_204902_6397477289918787376473.syserr
Total time: 440 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-7.x #342 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-7.x - Build # 342 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/342/
4 tests failed.
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 8 in https://127.0.0.1:37379/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 8 in https://127.0.0.1:37379/solr
at __randomizedtesting.SeedInfo.seed([672CDC9CFB164B10:A6DCA530D64681B7]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:885)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:599)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestUtilizeNode.test
Error Message:
no replica should be present in 127.0.0.1:36995_solr
Stack Trace:
java.lang.AssertionError: no replica should be present in 127.0.0.1:36995_solr
at __randomizedtesting.SeedInfo.seed([672CDC9CFB164B10:EF78E34655EA26E8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.TestUtilizeNode.test(TestUtilizeNode.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([672CDC9CFB164B10:3A64C21534D0ED5F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testSearchRate(TriggerIntegrationTest.java:1438)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testListeners
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([672CDC9CFB164B10:8AEC4B8C467AD6C9]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testListeners(TriggerIntegrationTest.java:1053)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12481 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> 1423870 INFO (SUITE-TriggerIntegrationTest-seed#[672CDC9CFB164B10]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_672CDC9CFB164B10-001/init-core-data-001
[junit4] 2> 1423870 WARN (SUITE-TriggerIntegrationTest-seed#[672CDC9CFB164B10]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=9 numCloses=9
[junit4] 2> 1423871 INFO (SUITE-TriggerIntegrationTest-seed#[672CDC9CFB164B10]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1423872 INFO (SUITE-TriggerIntegrationTest-seed#[672CDC9CFB164B10]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1423886 INFO (SUITE-TriggerIntegrationTest-seed#[672CDC9CFB164B10]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_672CDC9CFB164B10-001/tempDir-001
[junit4] 2> 1423886 INFO (SUITE-TriggerIntegrationTest-seed#[672CDC9CFB164B10]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1423923 INFO (Thread-1220) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1423924 INFO (Thread-1220) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1424030 INFO (SUITE-TriggerIntegrationTest-seed#[672CDC9CFB164B10]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:38623
[junit4] 2> 1424031 ERROR (Thread-1220) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1424209 INFO (zkConnectionManagerCallback-1555-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1424230 INFO (jetty-launcher-1552-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1424284 INFO (jetty-launcher-1552-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1424383 INFO (jetty-launcher-1552-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1424383 INFO (jetty-launcher-1552-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1424383 INFO (jetty-launcher-1552-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1424383 INFO (jetty-launcher-1552-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3366a1a4{/solr,null,AVAILABLE}
[junit4] 2> 1424385 INFO (jetty-launcher-1552-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@11046898{SSL,[ssl, http/1.1]}{127.0.0.1:33220}
[junit4] 2> 1424385 INFO (jetty-launcher-1552-thread-2) [ ] o.e.j.s.Server Started @1434485ms
[junit4] 2> 1424385 INFO (jetty-launcher-1552-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33220}
[junit4] 2> 1424385 ERROR (jetty-launcher-1552-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1424385 INFO (jetty-launcher-1552-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1424385 INFO (jetty-launcher-1552-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1424385 INFO (jetty-launcher-1552-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1424385 INFO (jetty-launcher-1552-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-30T11:21:25.165Z
[junit4] 2> 1424404 INFO (zkConnectionManagerCallback-1557-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1424404 INFO (jetty-launcher-1552-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1424456 INFO (jetty-launcher-1552-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1424456 INFO (jetty-launcher-1552-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1424456 INFO (jetty-launcher-1552-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1424469 INFO (jetty-launcher-1552-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@133a1e7c{/solr,null,AVAILABLE}
[junit4] 2> 1424469 INFO (jetty-launcher-1552-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@386d641b{SSL,[ssl, http/1.1]}{127.0.0.1:36491}
[junit4] 2> 1424469 INFO (jetty-launcher-1552-thread-1) [ ] o.e.j.s.Server Started @1434569ms
[junit4] 2> 1424469 INFO (jetty-launcher-1552-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36491}
[junit4] 2> 1424469 ERROR (jetty-launcher-1552-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1424469 INFO (jetty-launcher-1552-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1424469 INFO (jetty-launcher-1552-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1424469 INFO (jetty-launcher-1552-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1424469 INFO (jetty-launcher-1552-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-30T11:21:25.249Z
[junit4] 2> 1424659 INFO (zkConnectionManagerCallback-1559-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1424675 INFO (jetty-launcher-1552-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1424692 INFO (jetty-launcher-1552-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38623/solr
[junit4] 2> 1424784 INFO (jetty-launcher-1552-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38623/solr
[junit4] 2> 1424817 INFO (zkConnectionManagerCallback-1563-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1424835 INFO (zkConnectionManagerCallback-1567-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1424958 INFO (zkConnectionManagerCallback-1569-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1425011 INFO (zkConnectionManagerCallback-1571-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1425169 INFO (jetty-launcher-1552-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1425189 INFO (jetty-launcher-1552-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36491_solr
[junit4] 2> 1425206 INFO (jetty-launcher-1552-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.Overseer Overseer (id=72173305497321478-127.0.0.1:36491_solr-n_0000000000) starting
[junit4] 2> 1425315 INFO (jetty-launcher-1552-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36491_solr
[junit4] 2> 1425315 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1425317 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1425400 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: []
[junit4] 2> 1425400 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1425400 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1425401 INFO (zkCallback-1570-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1425402 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1425402 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1425403 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1425677 INFO (jetty-launcher-1552-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1425679 INFO (jetty-launcher-1552-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1425733 INFO (jetty-launcher-1552-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33220_solr
[junit4] 2> 1425735 INFO (jetty-launcher-1552-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36491.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1425761 INFO (zkCallback-1570-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1425790 INFO (jetty-launcher-1552-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36491.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1425791 INFO (jetty-launcher-1552-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36491.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1425792 INFO (jetty-launcher-1552-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_672CDC9CFB164B10-001/tempDir-001/node1/.
[junit4] 2> 1425870 INFO (zkCallback-1568-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1426224 INFO (jetty-launcher-1552-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33220.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1426238 INFO (jetty-launcher-1552-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33220.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1426239 INFO (jetty-launcher-1552-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33220.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1426252 INFO (jetty-launcher-1552-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_672CDC9CFB164B10-001/tempDir-001/node2/.
[junit4] 2> 1426405 DEBUG (ScheduledTrigger-3071-thread-2) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1426479 INFO (zkConnectionManagerCallback-1577-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1426514 INFO (zkConnectionManagerCallback-1581-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1426519 INFO (SUITE-TriggerIntegrationTest-seed#[672CDC9CFB164B10]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1426535 INFO (SUITE-TriggerIntegrationTest-seed#[672CDC9CFB164B10]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:38623/solr ready
[junit4] 2> 1426566 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventFromRestoredState
[junit4] 2> 1426583 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 1426583 DEBUG (zkCallback-1570-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1426584 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1426586 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1426586 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1426711 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1426873 INFO (qtp215693388-6533) [n:127.0.0.1:33220_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1427089 INFO (qtp1436487762-6548) [n:127.0.0.1:36491_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=36
[junit4] 2> 1427089 DEBUG (qtp215693388-6536) [n:127.0.0.1:33220_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1427106 INFO (qtp215693388-6536) [n:127.0.0.1:33220_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=287
[junit4] 2> 1427110 DEBUG (zkCallback-1570-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1427138 DEBUG (zkCallback-1570-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:36491_solr, 127.0.0.1:33220_solr]
[junit4] 2> 1427138 DEBUG (zkCallback-1570-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEFRS instantiated with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 1427146 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1427164 INFO (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1427164 INFO (qtp215693388-6538) [n:127.0.0.1:33220_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1427199 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1427199 DEBUG (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1427200 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1427200 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1427250 INFO (qtp215693388-6538) [n:127.0.0.1:33220_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=85
[junit4] 2> 1427251 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1427264 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1427264 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1427264 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1427265 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2cd62690{/solr,null,AVAILABLE}
[junit4] 2> 1427265 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@427d1b51{SSL,[ssl, http/1.1]}{127.0.0.1:44245}
[junit4] 2> 1427265 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.Server Started @1437366ms
[junit4] 2> 1427265 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44245}
[junit4] 2> 1427266 ERROR (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1427266 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1427266 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1427266 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1427266 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-30T11:21:28.046Z
[junit4] 2> 1427284 INFO (zkConnectionManagerCallback-1583-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1427287 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1427336 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38623/solr
[junit4] 2> 1427353 INFO (zkConnectionManagerCallback-1587-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1427366 INFO (zkConnectionManagerCallback-1589-thread-1-processing-n:127.0.0.1:44245_solr) [n:127.0.0.1:44245_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1427369 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:44245_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1427432 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:44245_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1427433 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:44245_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44245_solr
[junit4] 2> 1427447 INFO (zkCallback-1568-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1427470 INFO (zkCallback-1570-thread-2-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1427483 INFO (zkCallback-1588-thread-1-processing-n:127.0.0.1:44245_solr) [n:127.0.0.1:44245_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1427503 INFO (zkCallback-1580-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1427686 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:44245_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44245.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1427738 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:44245_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44245.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1427739 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:44245_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44245.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1427740 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:44245_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_672CDC9CFB164B10-001/tempDir-001/node3/.
[junit4] 2> 1428200 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1428200 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1428201 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:44245_solr at time 1765620973964947
[junit4] 2> 1429221 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1429221 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1429250 INFO (OverseerCollectionConfigSetProcessor-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1430226 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1430226 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1431226 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1431226 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1432248 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1432248 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1433248 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1433248 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1434254 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1434254 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1435263 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1435263 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1436263 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1436263 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1437268 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1437268 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1438278 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1438278 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1438278 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEFRS firing registered processor for nodes: [127.0.0.1:44245_solr] added at times [1765620973964947], now=1765631051167608
[junit4] 2> 1438278 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"645d2aaf60e93T3z02o1edokchnea5zc5ko0d1m",
[junit4] 2> "source":"node_added_triggerEFRS",
[junit4] 2> "eventTime":1765620973964947,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1765620973964947],
[junit4] 2> "nodeNames":["127.0.0.1:44245_solr"]}}
[junit4] 2> 1438282 DEBUG (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_triggerEFRS]
[junit4] 2> 1438283 INFO (ScheduledTrigger-3071-thread-1) [n:127.0.0.1:36491_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1438283 DEBUG (AutoscalingActionExecutor-3072-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"645d2aaf60e93T3z02o1edokchnea5zc5ko0d1m",
[junit4] 2> "source":"node_added_triggerEFRS",
[junit4] 2> "eventTime":1765620973964947,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1765620973964947],
[junit4] 2> "_enqueue_time_":1765631055412606,
[junit4] 2> "nodeNames":["127.0.0.1:44245_solr"]}}
[junit4] 2> 1438284 DEBUG (AutoscalingActionExecutor-3072-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerEFRS after 5000ms
[junit4] 2> 1438284 DEBUG (AutoscalingActionExecutor-3072-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=645d2aaf60e93T3z02o1edokchnea5zc5ko0d1m
[junit4] 2> 1438307 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1438379 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1438379 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1438379 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1438380 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@333c6842{/solr,null,AVAILABLE}
[junit4] 2> 1438380 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2cc277be{SSL,[ssl, http/1.1]}{127.0.0.1:38087}
[junit4] 2> 1438380 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.Server Started @1448481ms
[junit4] 2> 1438380 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38087}
[junit4] 2> 1438381 ERROR (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1438381 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1438381 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1438381 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1438381 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-30T11:21:39.161Z
[junit4] 2> 1438467 INFO (zkConnectionManagerCallback-1593-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1438475 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1438704 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38623/solr
[junit4] 2> 1438740 INFO (zkConnectionManagerCallback-1597-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1438741 INFO (zkConnectionManagerCallback-1599-thread-1-processing-n:127.0.0.1:38087_solr) [n:127.0.0.1:38087_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1438842 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:38087_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1438843 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:38087_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1438878 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:38087_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38087_solr
[junit4] 2> 1438879 INFO (zkCallback-1568-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1438882 INFO (zkCallback-1588-thread-1-processing-n:127.0.0.1:44245_solr) [n:127.0.0.1:44245_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1438882 INFO (zkCallback-1580-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1438890 INFO (zkCallback-1570-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1438934 INFO (zkCallback-1598-thread-1-processing-n:127.0.0.1:38087_solr) [n:127.0.0.1:38087_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1439653 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:38087_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38087.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1439690 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:38087_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38087.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1439690 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:38087_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38087.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4f991ba0
[junit4] 2> 1439707 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [n:127.0.0.1:38087_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_672CDC9CFB164B10-001/tempDir-001/node4/.
[junit4] 2> 1443290 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1443291 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1443291 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:38087_solr at time 1765636063986154
[junit4] 2> 1444291 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1444291 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1445291 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1445291 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1446296 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1446296 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1447297 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1447297 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1448302 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1448302 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1449302 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1449302 DEBUG (ScheduledTrigger-3071-thread-4) [n:127.0.0.1:36491_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1449958 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@386d641b{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1449958 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=777065249
[junit4] 2> 1449958 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1449958 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@9985749: rootName = solr_36491, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@8d37f25
[junit4] 2> 1449980 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1449980 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5bd5d1d5: rootName = solr_36491, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7cad63cb
[junit4] 2> 1449996 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1449996 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@58814df: rootName = solr_36491, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1a8ec394
[junit4] 2> 1449997 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:36491_solr
[junit4] 2> 1449997 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1449997 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1449997 INFO (zkCallback-1598-thread-1-processing-n:127.0.0.1:38087_solr) [n:127.0.0.1:38087_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1449998 INFO (zkCallback-1568-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1449998 INFO (zkCallback-1580-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1449998 INFO (zkCallback-1588-thread-1-processing-n:127.0.0.1:44245_solr) [n:127.0.0.1:44245_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1449998 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.Overseer Overseer (id=72173305497321478-127.0.0.1:36491_solr-n_0000000000) closing
[junit4] 2> 1449998 INFO (OverseerStateUpdate-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36491_solr
[junit4] 2> 1449998 WARN (OverseerAutoScalingTriggerThread-72173305497321478-127.0.0.1:36491_solr-n_0000000000) [n:127.0.0.1:36491_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1449998 INFO (zkCallback-1570-thread-1-processing-n:127.0.0.1:36491_solr) [n:127.0.0.1:36491_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 1449998 DEBUG (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1450000 INFO (zkCallback-1568-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33220_solr
[junit4] 2> 1450000 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@133a1e7c{/solr,null,UNAVAILABLE}
[junit4] 2> 1450000 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1450001 INFO (zkCallback-1568-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.Overseer Overseer (id=72173305497321477-127.0.0.1:33220_solr-n_0000000001) starting
[junit4] 2> 1450012 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1450016 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1450016 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44245_solr, 127.0.0.1:38087_solr, 127.0.0.1:33220_solr]
[junit4] 2> 1450016 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEFRS instantiated with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 1450017 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:44245_solr, 127.0.0.1:38087_solr, 127.0.0.1:33220_solr]
[junit4] 2> 1450017 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion -1
[junit4] 2> 1450017 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1450017 INFO (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1450017 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:38087_solr
[junit4] 2> 1450018 DEBUG (ScheduledTrigger-3092-thread-1) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1450018 DEBUG (ScheduledTrigger-3092-thread-1) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1450019 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1450019 DEBUG (ScheduledTrigger-3092-thread-1) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1451019 DEBUG (ScheduledTrigger-3092-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1451019 DEBUG (ScheduledTrigger-3092-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1451020 DEBUG (ScheduledTrigger-3092-thread-3) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1452020 DEBUG (ScheduledTrigger-3092-thread-1) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1452020 DEBUG (ScheduledTrigger-3092-thread-1) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1452020 DEBUG (ScheduledTrigger-3092-thread-4) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1453026 DEBUG (ScheduledTrigger-3092-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1453026 DEBUG (ScheduledTrigger-3092-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1453026 DEBUG (ScheduledTrigger-3092-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1454027 DEBUG (ScheduledTrigger-3092-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 1454027 DEBUG (ScheduledTrigger-3092-thread-4) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1454027 DEBUG (ScheduledTrigger-3092-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1454027 DEBUG (ScheduledTrigger-3092-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEFRS firing registered processor for nodes: [127.0.0.1:38087_solr] added at times [1765636063986154], now=1765646800193015
[junit4] 2> 1454027 DEBUG (ScheduledTrigger-3092-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"645d62e6581eaT3z02o1edokchnea5zc5ko0d1o",
[junit4] 2> "source":"node_added_triggerEFRS",
[junit4] 2> "eventTime":1765636063986154,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1765636063986154],
[junit4] 2> "nodeNames":["127.0.0.1:38087_solr"]}}
[junit4] 2> 1454035 DEBUG (ScheduledTrigger-3092-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_triggerEFRS]
[junit4] 2> 1454036 INFO (ScheduledTrigger-3092-thread-2) [n:127.0.0.1:33220_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1454036 DEBUG (AutoscalingActionExecutor-3093-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"645d62e6581eaT3z02o1edokchnea5zc5ko0d1o",
[junit4] 2> "source":"node_added_triggerEFRS",
[junit4] 2> "eventTime":1765636063986154,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1765636063986154],
[junit4] 2> "_enqueue_time_":1765646808964391,
[junit4] 2> "nodeNames":["127.0.0.1:38087_solr"]}}
[junit4] 2> 1454036 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[672CDC9CFB164B10]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEventFromRestoredState
[junit4] 2> 1454052 DEBUG (AutoscalingActionExecutor-3093-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms
[junit4] 2> 1454052 DEBUG (AutoscalingActionExecutor-3093-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerEFRS after 5000ms
[junit4] 2> 1454052 DEBUG (AutoscalingActionExecutor-3093-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 15 ms for event id=645d62e6581eaT3z02o1edokchnea5zc5ko0d1o
[junit4] 2> 1454125 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testContinueTriggersOnOverseerRestart
[junit4] 2> 1454125 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 5
[junit4] 2> 1454125 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1454125 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1454130 INFO (qtp215693388-6536) [n:127.0.0.1:33220_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1454165 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1454165 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1454166 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1454191 INFO (qtp215693388-6536) [n:127.0.0.1:33220_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=60
[junit4] 2> 1454212 INFO (qtp215693388-6539) [n:127.0.0.1:33220_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1454307 INFO (qtp1820037159-6630) [n:127.0.0.1:38087_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1454385 INFO (qtp1528959715-6601) [n:127.0.0.1:44245_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1454385 DEBUG (qtp215693388-6538) [n:127.0.0.1:33220_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1454385 INFO (qtp215693388-6538) [n:127.0.0.1:33220_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=193
[junit4] 2> 1454385 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1454386 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44245_solr, 127.0.0.1:38087_solr, 127.0.0.1:33220_solr]
[junit4] 2> 1454386 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerCTOOR instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 1454386 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1454387 INFO (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1454387 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1454387 DEBUG (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 1454388 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@11046898{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1454404 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=362747350
[junit4] 2> 1454405 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1454405 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@43f34488: rootName = solr_33220, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@25e82031
[junit4] 2> 1454410 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1454410 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@11e44dd5: rootName = solr_33220, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7cad63cb
[junit4] 2> 1454415 DEBUG (ScheduledTrigger-3092-thread-4) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1454415 DEBUG (ScheduledTrigger-3092-thread-4) [n:127.0.0.1:33220_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1454426 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1454426 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3d95d0a6: rootName = solr_33220, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1a8ec394
[junit4] 2> 1454427 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33220_solr
[junit4] 2> 1454427 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1454428 INFO (zkCallback-1568-thread-1-processing-n:127.0.0.1:33220_solr) [n:127.0.0.1:33220_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1454428 INFO (zkCallback-1588-thread-1-processing-n:127.0.0.1:44245_solr) [n:127.0.0.1:44245_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1454437 INFO (zkCallback-1580-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1454437 INFO (zkCallback-1598-thread-1-processing-n:127.0.0.1:38087_solr) [n:127.0.0.1:38087_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1454452 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1454452 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.Overseer Overseer (id=72173305497321477-127.0.0.1:33220_solr-n_0000000001) closing
[junit4] 2> 1454452 INFO (OverseerStateUpdate-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:33220_solr
[junit4] 2> 1454454 DEBUG (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1454452 WARN (OverseerAutoScalingTriggerThread-72173305497321477-127.0.0.1:33220_solr-n_0000000001) [n:127.0.0.1:33220_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1454456 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3366a1a4{/solr,null,UNAVAILABLE}
[junit4] 2> 1454456 INFO (zkCallback-1588-thread-1-processing-n:127.0.0.1:44245_solr) [n:127.0.0.1:44245_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44245_solr
[junit4] 2> 1454457 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1454458 INFO (zkCallback-1588-thread-1-processing-n:127.0.0.1:44245_solr) [n:127.0.0.1:44245_solr ] o.a.s.c.Overseer Overseer (id=72173305497321483-127.0.0.1:44245_solr-n_0000000002) starting
[junit4] 2> 1454461 DEBUG (OverseerAutoScalingTriggerThread-72173305497321483-127.0.0.1:44245_solr-n_0000000002) [n:127.0.0.1:44245_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1454469 DEBUG (OverseerAutoScalingTriggerThread-72173305497321483-127.0.0.1:44245_solr-n_0000000002) [n:127.0.0.1:44245_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1454469 DEBUG (OverseerAutoScalingTriggerThread-72173305497321483-127.0.0.1:44245_solr-n_0000000002) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44245_solr, 127.0.0.1:38087_solr]
[junit4] 2> 1454469 DEBUG (OverseerAutoScalingTriggerThread-72173305497321483-127.0.0.1:44245_solr-n_0000000002) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerCTOOR instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 1454470 DEBUG (OverseerAutoScalingTriggerThread-72173305497321483-127.0.0.1:44245_solr-n_0000000002) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:44245_solr, 127.0.0.1:38087_solr]
[junit4] 2> 1454470 DEBUG (OverseerAutoScalingTriggerThread-72173305497321483-127.0.0.1:44245_solr-n_0000000002) [n:127.0.0.1:44245_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 1454470 DEBUG (OverseerAutoScalingTriggerThread-72173305497321483-127.0.0.1:44245_solr-n_0000000002) [n:127.0.0.1:44245_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1454470 INFO (OverseerAutoScalingTriggerThread-72173305497321483-127.0.0.1:44245_solr-n_0000000002) [n:127.0.0.1:44245_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1454471 DEBUG (ScheduledTrigger-3097-thread-1) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1454471 DEBUG (ScheduledTrigger-3097-thread-1) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1454471 DEBUG (OverseerAutoScalingTriggerThread-72173305497321483-127.0.0.1:44245_solr-n_0000000002) [n:127.0.0.1:44245_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 1454472 DEBUG (ScheduledTrigger-3097-thread-1) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1455472 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1455472 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1455472 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1456472 DEBUG (ScheduledTrigger-3097-thread-3) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1456472 DEBUG (ScheduledTrigger-3097-thread-3) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1456472 DEBUG (ScheduledTrigger-3097-thread-1) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1457473 DEBUG (ScheduledTrigger-3097-thread-3) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1457473 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1457473 DEBUG (ScheduledTrigger-3097-thread-3) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1458473 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1458473 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1458473 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1459485 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1459485 DEBUG (ScheduledTrigger-3097-thread-4) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1459485 DEBUG (ScheduledTrigger-3097-thread-4) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1460485 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1460486 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1460486 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1461486 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1461486 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1461486 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1462499 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1462499 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1462499 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1463499 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1463499 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1463499 DEBUG (ScheduledTrigger-3097-thread-2) [n:127.0.0.1:44245_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1464459 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T08:27:37+11:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1464460 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1464460 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1464460 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1464460 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4bdff50a{/solr,null,AVAILABLE}
[junit4] 2> 1464461 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@25332282{SSL,[ssl, http/1.1]}{127.0.0.1:42207}
[junit4] 2> 1464461 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.e.j.s.Server Started @1474561ms
[junit4] 2> 1464461 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42207}
[junit4] 2> 1464461 ERROR (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1464461 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1464461 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1464461 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1464461 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-30T11:22:05.241Z
[junit4] 2> 1464471 INFO (zkConnectionManagerCallback-1603-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1464471 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1464478 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[672CDC9CFB164B10]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:38623/solr
[junit4] 2> 1464487
[...truncated too long message...]
0]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36387 36387
[junit4] 2> 3697074 INFO (Thread-4468) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36387 36387
[junit4] 2> 3697076 WARN (Thread-4468) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 34 /solr/configs/conf
[junit4] 2> 24 /solr/collections/tlog_replica_test_create_delete/terms/shard1
[junit4] 2> 19 /solr/collections/tlog_replica_test_create_delete/terms/shard2
[junit4] 2> 11 /solr/collections/tlog_replica_test_recovery/terms/shard1
[junit4] 2> 11 /solr/collections/tlog_replica_test_real_time_get/terms/shard1
[junit4] 2> 10 /solr/aliases.json
[junit4] 2> 10 /solr/collections/tlog_replica_test_add_docs/terms/shard1
[junit4] 2> 10 /solr/collections/tlog_replica_test_remove_leader/terms/shard1
[junit4] 2> 9 /solr/collections/tlog_replica_test_basic_leader_election/terms/shard1
[junit4] 2> 9 /solr/collections/tlog_replica_test_kill_leader/terms/shard1
[junit4] 2> 9 /solr/security.json
[junit4] 2> 8 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/terms/shard1
[junit4] 2> 7 /solr/collections/tlog_replica_test_delete_by_id/terms/shard1
[junit4] 2> 7 /solr/collections/tlog_replica_test_kill_tlog_replica/terms/shard1
[junit4] 2> 7 /solr/collections/tlog_replica_test_only_leader_indexes/terms/shard1
[junit4] 2> 4 /solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard1
[junit4] 2> 4 /solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 38 /solr/collections/tlog_replica_test_recovery/state.json
[junit4] 2> 29 /solr/collections/tlog_replica_test_create_delete/state.json
[junit4] 2> 28 /solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/state.json
[junit4] 2> 26 /solr/collections/tlog_replica_test_remove_leader/state.json
[junit4] 2> 26 /solr/collections/tlog_replica_test_kill_tlog_replica/state.json
[junit4] 2> 25 /solr/collections/tlog_replica_test_kill_leader/state.json
[junit4] 2> 24 /solr/collections/tlog_replica_test_add_remove_tlog_replica/state.json
[junit4] 2> 23 /solr/collections/tlog_replica_test_basic_leader_election/state.json
[junit4] 2> 15 /solr/collections/tlog_replica_test_real_time_get/state.json
[junit4] 2> 13 /solr/collections/tlog_replica_test_add_docs/state.json
[junit4] 2> 13 /solr/collections/tlog_replica_test_only_leader_indexes/state.json
[junit4] 2> 13 /solr/collections/tlog_replica_test_delete_by_id/state.json
[junit4] 2> 10 /solr/clusterprops.json
[junit4] 2> 10 /solr/clusterstate.json
[junit4] 2> 8 /solr/overseer_elect/election/72173445802098692-127.0.0.1:38230_solr-n_0000000000
[junit4] 2> 4 /solr/collections/tlog_replica_test_recovery/leader_elect/shard1/election/72173445802098692-core_node4-n_0000000000
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72173445802098692-core_node11-n_0000000001
[junit4] 2> 2 /solr/collections/tlog_replica_test_add_docs/leader_elect/shard1/election/72173445802098717-core_node6-n_0000000000
[junit4] 2> 2 /solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72173445802098717-core_node5-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/live_nodes
[junit4] 2> 10 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_672CDC9CFB164B10-001
[junit4] 2> Jan 30, 2018 11:59:18 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {foo=PostingsFormat(name=Direct), title_s=Lucene50(blocksize=128), id=FST50}, docValues:{_version_=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Lucene70), inplace_updatable_int=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=849, maxMBSortInHeap=7.923762605747803, sim=RandomSimilarity(queryNorm=false): {}, locale=es-DO, timezone=Africa/Monrovia
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=184182600,total=517472256
[junit4] 2> NOTE: All tests run in this JVM: [SystemInfoHandlerTest, TestNodeAddedTrigger, OverriddenZkACLAndCredentialsProvidersTest, TestPrepRecovery, TestIntervalFaceting, ClassificationUpdateProcessorFactoryTest, HdfsAutoAddReplicasIntegrationTest, OverseerTest, TestInfoStreamLogging, TestSchemalessBufferedUpdates, SignatureUpdateProcessorFactoryTest, SolrInfoBeanTest, TestGraphTermsQParserPlugin, DOMUtilTest, TestRequestStatusCollectionAPI, TestZkAclsWithHadoopAuth, SimpleCollectionCreateDeleteTest, TestClusterProperties, TestHdfsBackupRestoreCore, LeaderFailoverAfterPartitionTest, CoreAdminCreateDiscoverTest, SuggesterTest, DeleteStatusTest, AnalysisErrorHandlingTest, TestPerFieldSimilarity, TestJsonFacets, FullHLLTest, TemplateUpdateProcessorTest, TestLegacyNumericRangeQueryBuilder, TestSchemaResource, ConjunctionSolrSpellCheckerTest, TestClusterStateMutator, TestSolrCoreSnapshots, TestManagedResource, DistributedMLTComponentTest, TestDynamicFieldResource, TolerantUpdateProcessorTest, TestRandomCollapseQParserPlugin, ShardRoutingCustomTest, TestZkChroot, TestPKIAuthenticationPlugin, TestCloudManagedSchema, SharedFSAutoReplicaFailoverTest, SampleTest, V2ApiIntegrationTest, PeerSyncTest, HdfsUnloadDistributedZkTest, SOLR749Test, CollectionsAPIAsyncDistributedZkTest, TestSolrConfigHandlerCloud, FileUtilsTest, TestFastLRUCache, TestNonDefinedSimilarityFactory, TestCharFilters, CdcrBidirectionalTest, TestEmbeddedSolrServerAdminHandler, SuggestComponentContextFilterQueryTest, TestCustomStream, HdfsChaosMonkeyNothingIsSafeTest, ExitableDirectoryReaderTest, TestXmlQParserPlugin, TestSimpleQParserPlugin, HdfsRecoverLeaseTest, SoftAutoCommitTest, SchemaVersionSpecificBehaviorTest, TestApiFramework, TestHighFrequencyDictionaryFactory, ManagedSchemaRoundRobinCloudTest, TestElisionMultitermQuery, SecurityConfHandlerTest, TestExpandComponent, ReplicaListTransformerTest, TestTrie, IndexSchemaTest, TestStressUserVersions, MetricsConfigTest, TestXIncludeConfig, ConfigSetsAPITest, ConfigureRecoveryStrategyTest, LoggingHandlerTest, CreateRoutedAliasTest, TestLeaderElectionWithEmptyReplica, TestNumericTokenStream, CustomCollectionTest, BadComponentTest, CollectionStateFormat2Test, TestSubQueryTransformer, TestDocTermOrds, TestScoreJoinQPScore, SortByFunctionTest, TestCollapseQParserPlugin, AddSchemaFieldsUpdateProcessorFactoryTest, TestDistributedGrouping, SliceStateTest, SimpleMLTQParserTest, DistributedFacetPivotLongTailTest, AssignTest, UtilsToolTest, PolyFieldTest, TestSolrConfigHandler, BasicAuthStandaloneTest, TestReloadAndDeleteDocs, TestSimpleTrackingShardHandler, QueryEqualityTest, TestCoreAdminApis, HttpPartitionTest, TestHdfsCloudBackupRestore, TestCloudInspectUtil, TestBinaryResponseWriter, NodeAddedTriggerTest, SolrSlf4jReporterTest, BlobRepositoryCloudTest, TestRequestForwarding, TestRandomFlRTGCloud, SyncSliceTest, TestStressRecovery, HdfsSyncSliceTest, TestDelegationWithHadoopAuth, DistributedExpandComponentTest, InfoHandlerTest, TestHalfAndHalfDocValues, DeleteNodeTest, TestSearchPerf, TestSimDistributedQueue, PrimUtilsTest, TestBM25SimilarityFactory, TestTrackingShardHandlerFactory, TestRetrieveFieldsOptimizer, ChaosMonkeySafeLeaderTest, OpenCloseCoreStressTest, ShardRoutingTest, BasicZkTest, RecoveryZkTest, ClusterStateUpdateTest, TestRandomFaceting, ZkSolrClientTest, ZkCLITest, TestFaceting, TestRecovery, TestStressReorder, TestReload, TestCoreContainer, SimpleFacetsTest, SolrCoreTest, QueryElevationComponentTest, TestFiltering, DirectUpdateHandlerTest, TestBadConfig, TestIndexSearcher, DistributedQueryElevationComponentTest, TestExtendedDismaxParser, TestFoldingMultitermQuery, TestStressLucene, SpatialFilterTest, TestCSVLoader, WordBreakSolrSpellCheckerTest, SolrCoreCheckLockOnStartupTest, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, DebugComponentTest, CacheHeaderTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, TestQueryUtils, TestWriterPerf, TestQueryTypes, FileBasedSpellCheckerTest, XmlUpdateRequestHandlerTest, DocumentBuilderTest, PathHierarchyTokenizerFactoryTest, RequiredFieldsTest, TestSolrQueryParser, UpdateRequestProcessorFactoryTest, TestCSVResponseWriter, UniqFieldsUpdateProcessorFactoryTest, BinaryUpdateRequestHandlerTest, PingRequestHandlerTest, TestLFUCache, PreAnalyzedFieldTest, TestSystemIdResolver, TestSuggestSpellingConverter, DateFieldTest, TestSolrJ, TestLRUCache, ZkNodePropsTest, UUIDFieldTest, CircularListTest, TestRTGBase, CursorPagingTest, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestEmbeddedSolrServerConstructors, TestJettySolrRunner, ActionThrottleTest, AsyncCallRequestStatusResponseTest, CleanupOldIndexTest, DeleteInactiveReplicaTest, DistribJoinFromCollectionTest, LegacyCloudClusterPropTest, MoveReplicaHDFSFailoverTest, RestartWhileUpdatingTest, RollingRestartTest, TestCloudDeleteByQuery, TestCloudRecovery, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestCryptoKeys, TestExclusionRuleCollectionAccess, TestPullReplica, TestTlogReplica]
[junit4] Completed [579/772 (3!)] on J1 in 133.63s, 13 tests, 1 failure <<< FAILURES!
[...truncated 48768 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 341 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/341/
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([29181B033C1F277C:93142C8C63F7F133]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.junit.Assert.assertNull(Assert.java:562)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<8> but was:<10>
Stack Trace:
java.lang.AssertionError: expected:<8> but was:<10>
at __randomizedtesting.SeedInfo.seed([29181B033C1F277C:44E4BFFE8657D87B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:259)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12845 lines...]
[junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
[junit4] 2> 2288060 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[29181B033C1F277C]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/init-core-data-001
[junit4] 2> 2288060 WARN (SUITE-AutoscalingHistoryHandlerTest-seed#[29181B033C1F277C]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 2288061 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[29181B033C1F277C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2288063 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[29181B033C1F277C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 2288063 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[29181B033C1F277C]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001
[junit4] 2> 2288063 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[29181B033C1F277C]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2288077 INFO (Thread-2300) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2288077 INFO (Thread-2300) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2288151 ERROR (Thread-2300) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2288187 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[29181B033C1F277C]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:33065
[junit4] 2> 2288284 INFO (zkConnectionManagerCallback-2613-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2288350 INFO (jetty-launcher-2610-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2288352 INFO (jetty-launcher-2610-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2288395 INFO (jetty-launcher-2610-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2288474 INFO (jetty-launcher-2610-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2288474 INFO (jetty-launcher-2610-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2288475 INFO (jetty-launcher-2610-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2288495 INFO (jetty-launcher-2610-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2aea3cc0{/solr,null,AVAILABLE}
[junit4] 2> 2288496 INFO (jetty-launcher-2610-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@14aebed9{SSL,[ssl, http/1.1]}{127.0.0.1:36656}
[junit4] 2> 2288496 INFO (jetty-launcher-2610-thread-1) [ ] o.e.j.s.Server Started @2297602ms
[junit4] 2> 2288496 INFO (jetty-launcher-2610-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36656}
[junit4] 2> 2288496 ERROR (jetty-launcher-2610-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2288497 INFO (jetty-launcher-2610-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2288497 INFO (jetty-launcher-2610-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2288497 INFO (jetty-launcher-2610-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2288497 INFO (jetty-launcher-2610-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T21:17:32.397Z
[junit4] 2> 2288535 INFO (jetty-launcher-2610-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2288535 INFO (jetty-launcher-2610-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2288535 INFO (jetty-launcher-2610-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2288547 INFO (jetty-launcher-2610-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6828e06b{/solr,null,AVAILABLE}
[junit4] 2> 2288548 INFO (jetty-launcher-2610-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@229aca96{SSL,[ssl, http/1.1]}{127.0.0.1:45490}
[junit4] 2> 2288548 INFO (jetty-launcher-2610-thread-2) [ ] o.e.j.s.Server Started @2297654ms
[junit4] 2> 2288548 INFO (jetty-launcher-2610-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45490}
[junit4] 2> 2288548 ERROR (jetty-launcher-2610-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2288548 INFO (jetty-launcher-2610-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2288548 INFO (jetty-launcher-2610-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2288548 INFO (jetty-launcher-2610-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2288548 INFO (jetty-launcher-2610-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T21:17:32.448Z
[junit4] 2> 2288568 INFO (zkConnectionManagerCallback-2617-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2288568 INFO (jetty-launcher-2610-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2288614 INFO (jetty-launcher-2610-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2288614 INFO (jetty-launcher-2610-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2288614 INFO (jetty-launcher-2610-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2288614 INFO (jetty-launcher-2610-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19bce01d{/solr,null,AVAILABLE}
[junit4] 2> 2288615 INFO (jetty-launcher-2610-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1c9d1a2f{SSL,[ssl, http/1.1]}{127.0.0.1:40827}
[junit4] 2> 2288615 INFO (jetty-launcher-2610-thread-3) [ ] o.e.j.s.Server Started @2297721ms
[junit4] 2> 2288615 INFO (jetty-launcher-2610-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40827}
[junit4] 2> 2288615 ERROR (jetty-launcher-2610-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2288615 INFO (jetty-launcher-2610-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 2288615 INFO (jetty-launcher-2610-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2288615 INFO (jetty-launcher-2610-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2288615 INFO (jetty-launcher-2610-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T21:17:32.515Z
[junit4] 2> 2288649 INFO (zkConnectionManagerCallback-2615-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2288649 INFO (zkConnectionManagerCallback-2619-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2288650 INFO (jetty-launcher-2610-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2288661 INFO (jetty-launcher-2610-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2288801 INFO (jetty-launcher-2610-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33065/solr
[junit4] 2> 2288809 INFO (zkConnectionManagerCallback-2623-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2288945 INFO (jetty-launcher-2610-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33065/solr
[junit4] 2> 2288979 INFO (jetty-launcher-2610-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33065/solr
[junit4] 2> 2288993 INFO (zkConnectionManagerCallback-2629-thread-1-processing-n:127.0.0.1:45490_solr) [n:127.0.0.1:45490_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2289046 INFO (zkConnectionManagerCallback-2627-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2289114 INFO (zkConnectionManagerCallback-2633-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2289115 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100663894330007, likely client has closed socket
[junit4] 2> 2289167 INFO (zkConnectionManagerCallback-2636-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2289192 INFO (zkConnectionManagerCallback-2638-thread-1-processing-n:127.0.0.1:36656_solr) [n:127.0.0.1:36656_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2289225 INFO (jetty-launcher-2610-thread-3) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2289225 INFO (jetty-launcher-2610-thread-3) [n:127.0.0.1:40827_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40827_solr
[junit4] 2> 2289226 INFO (jetty-launcher-2610-thread-3) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer Overseer (id=72169987228499976-127.0.0.1:40827_solr-n_0000000000) starting
[junit4] 2> 2289313 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 2289313 INFO (jetty-launcher-2610-thread-3) [n:127.0.0.1:40827_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40827_solr
[junit4] 2> 2289329 DEBUG (OverseerAutoScalingTriggerThread-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 2289402 DEBUG (OverseerAutoScalingTriggerThread-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 2289443 DEBUG (OverseerAutoScalingTriggerThread-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: []
[junit4] 2> 2289443 DEBUG (OverseerAutoScalingTriggerThread-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 2289443 DEBUG (OverseerAutoScalingTriggerThread-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 2289483 INFO (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2289556 DEBUG (OverseerAutoScalingTriggerThread-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2289557 DEBUG (OverseerAutoScalingTriggerThread-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 2289557 INFO (jetty-launcher-2610-thread-1) [n:127.0.0.1:36656_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2289558 DEBUG (ScheduledTrigger-5179-thread-1) [n:127.0.0.1:40827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 2289559 INFO (jetty-launcher-2610-thread-1) [n:127.0.0.1:36656_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2289572 INFO (jetty-launcher-2610-thread-1) [n:127.0.0.1:36656_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36656_solr
[junit4] 2> 2289645 INFO (zkCallback-2635-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2289675 INFO (zkCallback-2637-thread-1-processing-n:127.0.0.1:36656_solr) [n:127.0.0.1:36656_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2289678 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:36656_solr"} current state version: 0
[junit4] 2> 2289678 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:36656_solr
[junit4] 2> 2289747 INFO (jetty-launcher-2610-thread-3) [n:127.0.0.1:40827_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40827.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad6fe1a
[junit4] 2> 2289770 INFO (jetty-launcher-2610-thread-3) [n:127.0.0.1:40827_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40827.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad6fe1a
[junit4] 2> 2289770 INFO (jetty-launcher-2610-thread-3) [n:127.0.0.1:40827_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40827.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad6fe1a
[junit4] 2> 2289783 INFO (jetty-launcher-2610-thread-3) [n:127.0.0.1:40827_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/.
[junit4] 2> 2289937 INFO (jetty-launcher-2610-thread-1) [n:127.0.0.1:36656_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36656.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad6fe1a
[junit4] 2> 2289982 INFO (jetty-launcher-2610-thread-1) [n:127.0.0.1:36656_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36656.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad6fe1a
[junit4] 2> 2289983 INFO (jetty-launcher-2610-thread-1) [n:127.0.0.1:36656_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36656.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad6fe1a
[junit4] 2> 2289984 INFO (jetty-launcher-2610-thread-1) [n:127.0.0.1:36656_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/.
[junit4] 2> 2290036 INFO (jetty-launcher-2610-thread-2) [n:127.0.0.1:45490_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2290037 INFO (jetty-launcher-2610-thread-2) [n:127.0.0.1:45490_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2290038 INFO (jetty-launcher-2610-thread-2) [n:127.0.0.1:45490_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45490_solr
[junit4] 2> 2290054 INFO (zkCallback-2635-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2290054 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:45490_solr"} current state version: 0
[junit4] 2> 2290054 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:45490_solr
[junit4] 2> 2290091 INFO (zkCallback-2637-thread-1-processing-n:127.0.0.1:36656_solr) [n:127.0.0.1:36656_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2290125 INFO (zkCallback-2628-thread-1-processing-n:127.0.0.1:45490_solr) [n:127.0.0.1:45490_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2290238 INFO (jetty-launcher-2610-thread-2) [n:127.0.0.1:45490_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45490.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad6fe1a
[junit4] 2> 2290260 INFO (jetty-launcher-2610-thread-2) [n:127.0.0.1:45490_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45490.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad6fe1a
[junit4] 2> 2290260 INFO (jetty-launcher-2610-thread-2) [n:127.0.0.1:45490_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45490.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ad6fe1a
[junit4] 2> 2290274 INFO (jetty-launcher-2610-thread-2) [n:127.0.0.1:45490_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/.
[junit4] 2> 2290396 INFO (zkConnectionManagerCallback-2645-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2290424 INFO (zkConnectionManagerCallback-2649-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2290425 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[29181B033C1F277C]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2290425 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[29181B033C1F277C]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33065/solr ready
[junit4] 2> 2290520 INFO (qtp208076172-12919) [n:127.0.0.1:36656_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2290534 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
[junit4] 2> 2290535 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/protwords.txt
[junit4] 2> 2290536 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
[junit4] 2> 2290553 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/solrconfig.xml
[junit4] 2> 2290555 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/synonyms.txt
[junit4] 2> 2290569 DEBUG (ScheduledTrigger-5179-thread-1) [n:127.0.0.1:40827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2290569 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/stopwords.txt
[junit4] 2> 2290570 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/params.json
[junit4] 2> 2290571 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_it.txt
[junit4] 2> 2290572 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_no.txt
[junit4] 2> 2290592 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hy.txt
[junit4] 2> 2290593 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stemdict_nl.txt
[junit4] 2> 2290606 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ca.txt
[junit4] 2> 2290607 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_id.txt
[junit4] 2> 2290608 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ga.txt
[junit4] 2> 2290609 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_da.txt
[junit4] 2> 2290626 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hi.txt
[junit4] 2> 2290627 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ja.txt
[junit4] 2> 2290628 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_pt.txt
[junit4] 2> 2290629 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hu.txt
[junit4] 2> 2290629 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_el.txt
[junit4] 2> 2290642 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ru.txt
[junit4] 2> 2290643 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_tr.txt
[junit4] 2> 2290644 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ar.txt
[junit4] 2> 2290644 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/userdict_ja.txt
[junit4] 2> 2290645 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_it.txt
[junit4] 2> 2290662 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_eu.txt
[junit4] 2> 2290663 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_cz.txt
[junit4] 2> 2290664 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stoptags_ja.txt
[junit4] 2> 2290665 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_fr.txt
[junit4] 2> 2290666 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fa.txt
[junit4] 2> 2290679 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_de.txt
[junit4] 2> 2290679 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ga.txt
[junit4] 2> 2290680 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ca.txt
[junit4] 2> 2290681 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_nl.txt
[junit4] 2> 2290682 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_bg.txt
[junit4] 2> 2290699 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_es.txt
[junit4] 2> 2290699 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_en.txt
[junit4] 2> 2290700 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_lv.txt
[junit4] 2> 2290701 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_th.txt
[junit4] 2> 2290701 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ro.txt
[junit4] 2> 2290702 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fr.txt
[junit4] 2> 2290715 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_sv.txt
[junit4] 2> 2290716 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fi.txt
[junit4] 2> 2290717 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/hyphenations_ga.txt
[junit4] 2> 2290717 INFO (OverseerThreadFactory-5181-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_gl.txt
[junit4] 2> 2290754 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "numShards":"1",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 2290754 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
[junit4] 2> 2290754 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
[junit4] 2> 2290846 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2290846 INFO (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2290847 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 2290847 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 2290848 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:36656/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2290848 INFO (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:36656/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2290864 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 2290864 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:36656/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 2290881 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40827/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 2290881 INFO (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40827/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2290882 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:36656/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 2290882 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:36656/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:40827/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 2290986 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
[junit4] 2> 2291425 INFO (qtp208076172-12923) [n:127.0.0.1:36656_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2291425 INFO (qtp208076172-12923) [n:127.0.0.1:36656_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2291614 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:36656/solr",
[junit4] 2> "node_name":"127.0.0.1:36656_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2291614 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:36656/solr",
[junit4] 2> "node_name":"127.0.0.1:36656_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 2291614 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"https://127.0.0.1:36656/solr","node_name":"127.0.0.1:36656_solr","state":"down","type":"NRT"}
[junit4] 2> 2291614 INFO (qtp809034762-12932) [n:127.0.0.1:45490_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2291614 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:36656/solr",
[junit4] 2> "node_name":"127.0.0.1:36656_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:40827/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 2291615 INFO (qtp809034762-12932) [n:127.0.0.1:45490_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2291724 DEBUG (ScheduledTrigger-5179-thread-2) [n:127.0.0.1:40827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2291740 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
[junit4] 2> 2291761 INFO (qtp1377347791-12944) [n:127.0.0.1:40827_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n4&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2291761 INFO (qtp1377347791-12944) [n:127.0.0.1:40827_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2291768 INFO (zkCallback-2637-thread-1-processing-n:127.0.0.1:36656_solr) [n:127.0.0.1:36656_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2291785 INFO (zkCallback-2637-thread-2-processing-n:127.0.0.1:36656_solr) [n:127.0.0.1:36656_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2291869 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:40827/solr",
[junit4] 2> "node_name":"127.0.0.1:40827_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2291869 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:40827/solr",
[junit4] 2> "node_name":"127.0.0.1:40827_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 2291870 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4","base_url":"https://127.0.0.1:40827/solr","node_name":"127.0.0.1:40827_solr","state":"down","type":"NRT"}
[junit4] 2> 2291870 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/2)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:36656/solr",
[junit4] 2> "node_name":"127.0.0.1:36656_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:40827/solr",
[junit4] 2> "node_name":"127.0.0.1:40827_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 2291906 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "node_name":"127.0.0.1:45490_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 2291906 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "node_name":"127.0.0.1:45490_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 2291906 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"https://127.0.0.1:45490/solr","node_name":"127.0.0.1:45490_solr","state":"down","type":"NRT"}
[junit4] 2> 2291906 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/2)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:45490/solr",
[junit4] 2> "node_name":"127.0.0.1:45490_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"https://127.0.0.1:36656/solr",
[junit4] 2> "node_name":"127.0.0.1:36656_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"https://127.0.0.1:40827/solr",
[junit4] 2> "node_name":"127.0.0.1:40827_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 2292035 DEBUG (OverseerStateUpdate-72169987228499976-127.0.0.1:40827_solr-n_0000000000) [n:127.0.0.1:40827_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 2
[junit4] 2> 2292036 INFO (zkCallback-2628-thread-1-processing-n:127.0.0.1:45490_solr) [n:127.0.0.1:45490_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2292044 INFO (zkCallback-2635-thread-1-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2292048 INFO (zkCallback-2637-thread-2-processing-n:127.0.0.1:36656_solr) [n:127.0.0.1:36656_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2292048 INFO (zkCallback-2637-thread-1-processing-n:127.0.0.1:36656_solr) [n:127.0.0.1:36656_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2292049 INFO (zkCallback-2635-thread-2-processing-n:127.0.0.1:40827_solr) [n:127.0.0.1:40827_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2292049 INFO (zkCallback-2628-thread-2-processing-n:127.0.0.1:45490_solr) [n:127.0.0.1:45490_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2292502 INFO (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 2292503 INFO (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 2292503 WARN (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
[junit4] 2> 2292539 WARN (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 2292539 WARN (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib
[junit4] 2> 2292539 WARN (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 2292539 WARN (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib
[junit4] 2> 2292540 WARN (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 2292540 WARN (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib
[junit4] 2> 2292540 WARN (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 2292576 INFO (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2292830 DEBUG (ScheduledTrigger-5179-thread-1) [n:127.0.0.1:40827_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2292895 INFO (qtp208076172-12923) [n:127.0.0.1:36656_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.s.IndexSchema [AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] Schema name=default-config
[junit4] 2> 2293218 INFO (qtp809034762-12932) [n:127.0.0.1:45490_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 2293219 INFO (qtp809034762-12932) [n:127.0.0.1:45490_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 2293219 WARN (qtp809034762-12932) [n:127.0.0.1:45490_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
[junit4] 2> 2293219 WARN (qtp809034762-12932) [n:127.0.0.1:45490_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 2293220 WARN (qtp809034762-12932) [n:127.0.0.1:45490_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
[junit4] 2> 2293220 WARN (qtp809034762-12932) [n:127.0.0.1:45490_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 2293220 WARN (qtp809034762-12932) [n:127.0.0.1:45490_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib
[junit4] 2> 2293220 WARN (qtp809034762-12932) [n:127.0.0.1:45490_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 2293220 WARN (qtp809034762-12932) [n:127.0.0.1:45490_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib
[junit4] 2> 2293220 WARN (qtp809034762-12932) [n:127.0.0.1:45490_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 2293292 INFO (qtp809034762-12932) [n:127.0.0.1:45490_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2293393 INFO (qtp1377347791-12944) [n:127.0.0.1:40827_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 2293393 INFO (qtp1377347791-12944) [n:127.0.0.1:40827_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 2293393 WARN (qtp1377347791-12944) [n:127.0.0.1:40827_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/extraction/lib
[junit4] 2> 2293394 WARN (qtp1377347791-12944) [n:127.0.0.1:40827_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
[junit4] 2> 2293394 WARN (qtp1377347791-12944) [n:127.0.0.1:40827_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/clustering/lib
[junit4] 2> 2293394 WARN (qtp1377347791-12944) [n:127.0.0.1:40827_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
[junit4] 2> 2293394 WARN (qtp1377347791-12944) [n:127.0.0.1:40827_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../contrib/langid/lib
[junit4] 2> 2293394 WARN (qtp1377347791-12944) [n:127.0.0.1:40827_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n4] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_29181B033C1F277C-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n4/../../../../dist
[junit4] 2> 2293394 WARN (qtp1377347791-12944) [n:127.0.0.1:40827_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r
[...truncated too long message...]
[junit4] 2> 3851341 INFO (jetty-closer-4880-thread-4) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3851341 INFO (jetty-closer-4880-thread-9) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@40d2346a{/solr,null,UNAVAILABLE}
[junit4] 2> 3851343 INFO (jetty-closer-4880-thread-9) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3851343 ERROR (SUITE-TriggerIntegrationTest-seed#[29181B033C1F277C]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3851344 INFO (SUITE-TriggerIntegrationTest-seed#[29181B033C1F277C]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36134 36134
[junit4] 2> 3851614 INFO (Thread-9125) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36134 36134
[junit4] 2> 3851618 WARN (Thread-9125) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 36 /solr/collections/collection1/terms/shard1
[junit4] 2> 22 /solr/clusterprops.json
[junit4] 2> 22 /solr/aliases.json
[junit4] 2> 21 /solr/security.json
[junit4] 2> 6 /solr/configs/conf
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 104 /solr/collections/collection1/state.json
[junit4] 2> 22 /solr/clusterstate.json
[junit4] 2> 12 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/72170076711092238-core_node8-n_0000000003
[junit4] 2> 2 /solr/overseer_elect/election/72170076711092278-127.0.0.1:40115_solr-n_0000000016
[junit4] 2> 2 /solr/overseer_elect/election/72170076711092269-127.0.0.1:45580_solr-n_0000000013
[junit4] 2> 2 /solr/overseer_elect/election/72170076711092254-127.0.0.1:40667_solr-n_0000000008
[junit4] 2> 2 /solr/overseer_elect/election/72170076711092281-127.0.0.1:37132_solr-n_0000000017
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/72170076711092235-core_node4-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/72170076711092241-127.0.0.1:37178_solr-n_0000000004
[junit4] 2> 2 /solr/overseer_elect/election/72170076711092263-127.0.0.1:34082_solr-n_0000000011
[junit4] 2> 2 /solr/overseer_elect/election/72170076711092272-127.0.0.1:39568_solr-n_0000000014
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/live_nodes
[junit4] 2> 22 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/queue
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_29181B033C1F277C-001
[junit4] 2> Jan 29, 2018 9:43:34 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {_version_=PostingsFormat(name=Memory), id=Lucene50(blocksize=128), x_s=PostingsFormat(name=Memory)}, docValues:{}, maxPointsInLeafNode=470, maxMBSortInHeap=5.967889822050157, sim=RandomSimilarity(queryNorm=false): {}, locale=ar-OM, timezone=Asia/Chita
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=79976520,total=494403584
[junit4] 2> NOTE: All tests run in this JVM: [TestShardHandlerFactory, HttpPartitionTest, CollectionStateFormat2Test, PreAnalyzedUpdateProcessorTest, HttpSolrCallGetCoreTest, TestHdfsUpdateLog, HdfsThreadLeakTest, TestSolrFieldCacheBean, SimpleCollectionCreateDeleteTest, TestSimDistributedQueue, ClassificationUpdateProcessorIntegrationTest, UpdateRequestProcessorFactoryTest, SliceStateTest, TestCustomDocTransformer, TestSmileRequest, TestFieldCacheSortRandom, BlockCacheTest, DistributedFacetSimpleRefinementLongTailTest, RequiredFieldsTest, PreAnalyzedFieldManagedSchemaCloudTest, TestClusterStateProvider, MetricsHandlerTest, BlockJoinFacetSimpleTest, TestNodeAddedTrigger, PrimUtilsTest, SuggestComponentTest, DeleteStatusTest, TestRequestForwarding, CreateRoutedAliasTest, TestManagedSchema, CustomCollectionTest, TestRuleBasedAuthorizationPlugin, TestFoldingMultitermQuery, V2StandaloneTest, TestHttpShardHandlerFactory, AutoScalingHandlerTest, TestImpersonationWithHadoopAuth, BitVectorTest, TestSolrQueryResponse, AnalyticsQueryTest, TestTlogReplica, QueryElevationComponentTest, CdcrReplicationHandlerTest, TestPartialUpdateDeduplication, TestMaxTokenLenTokenizer, AssignTest, TestQueryUtils, FastVectorHighlighterTest, TestEmbeddedSolrServerAdminHandler, LeaderFailoverAfterPartitionTest, TermsComponentTest, SignatureUpdateProcessorFactoryTest, HdfsDirectoryFactoryTest, DocumentBuilderTest, TestLockTree, SolrCoreTest, TestSimpleTrackingShardHandler, DirectUpdateHandlerOptimizeTest, JSONWriterTest, ZkStateReaderTest, TestRawResponseWriter, SolrMetricsIntegrationTest, TestFieldTypeResource, DeleteNodeTest, TestLegacyFieldCache, XsltUpdateRequestHandlerTest, OverseerRolesTest, ScriptEngineTest, SampleTest, TestUniqueKeyFieldResource, QueryEqualityTest, AsyncCallRequestStatusResponseTest, TestXIncludeConfig, TestCollectionAPIs, DocValuesMissingTest, SolrXmlInZkTest, OverseerTest, LoggingHandlerTest, TestMaxScoreQueryParser, TestLegacyNumericUtils, TestStreamBody, HdfsRecoveryZkTest, TestMinMaxOnMultiValuedField, SoftAutoCommitTest, DistributedQueryComponentCustomSortTest, ShardRoutingCustomTest, VMParamsZkACLAndCredentialsProvidersTest, SpellCheckCollatorWithCollapseTest, DistributedFacetExistsSmallTest, TestManagedResourceStorage, HighlighterTest, BJQParserTest, FacetPivotSmallTest, TestLRUCache, NodeAddedTriggerTest, SolrInfoBeanTest, TestSolrJ, TestGraphMLResponseWriter, SortSpecParsingTest, LargeFieldTest, SharedFSAutoReplicaFailoverTest, DirectSolrConnectionTest, PrimitiveFieldTypeTest, TestExecutePlanAction, TestRandomFlRTGCloud, UpdateLogTest, SolrCloudReportersTest, TestCloudPseudoReturnFields, TestFunctionQuery, JavabinLoaderTest, MultiThreadedOCPTest, SystemInfoHandlerTest, TestConfigOverlay, TestJettySolrRunner, DistributedIntervalFacetingTest, IndexSchemaTest, TestUninvertingReader, TestPostingsSolrHighlighter, SolrCoreMetricManagerTest, TestZkChroot, HdfsBasicDistributedZk2Test, ShufflingReplicaListTransformerTest, TestJsonRequest, AlternateDirectoryTest, TestShortCircuitedRequests, RequestLoggingTest, TestStressVersions, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, SyncSliceTest, OpenCloseCoreStressTest, LeaderElectionIntegrationTest, ShardRoutingTest, BasicZkTest, TestReplicationHandler, LeaderElectionTest, ZkSolrClientTest, TestDistributedGrouping, TestFaceting, TestRecovery, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestStressReorder, TestJoin, TestReload, SimpleFacetsTest, TestSolr4Spatial, SolrCmdDistributorTest, PeerSyncTest, BasicFunctionalityTest, TestBadConfig, TestIndexSearcher, DistributedQueryElevationComponentTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, TestExtendedDismaxParser, SuggesterFSTTest, CoreAdminHandlerTest, SolrRequestParserTest, SuggesterTSTTest, SuggesterTest, TestTrie, SpatialFilterTest, SuggesterWFSTTest, PolyFieldTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, TestUpdate, FieldMutatingUpdateProcessorTest, TestAtomicUpdateErrorCases, TestWordDelimiterFilterFactory, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, DocValuesMultiTest, DebugComponentTest, TestSurroundQueryParser, LukeRequestHandlerTest, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, TestQueryTypes, FileBasedSpellCheckerTest, XmlUpdateRequestHandlerTest, PathHierarchyTokenizerFactoryTest, TermVectorComponentTest, TestSolrQueryParser, IndexSchemaRuntimeFieldTest, RegexBoostProcessorTest, SolrPluginUtilsTest, TestJmxIntegration, TestCollationField, QueryParsingTest, TestAnalyzedSuggestions, JsonLoaderTest, BinaryUpdateRequestHandlerTest, CSVRequestHandlerTest, SearchHandlerTest, TestLFUCache, TestStressRecovery, TestTolerantSearch, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, ConnectionReuseTest, AssignBackwardCompatibilityTest, CleanupOldIndexTest, ConcurrentCreateRoutedAliasTest, ConfigSetsAPITest, DeleteLastCustomShardedReplicaTest, DeleteReplicaTest, DistribJoinFromCollectionTest, DistributedQueueTest, LeaderElectionContextKeyTest, LegacyCloudClusterPropTest, MoveReplicaHDFSFailoverTest, MoveReplicaHDFSTest, RollingRestartTest, TestCloudDeleteByQuery, TestCloudInspectUtil, TestCloudPivotFacet, TestClusterProperties, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestDeleteCollectionOnDownNodes, TestDistribDocBasedVersion, TestLeaderInitiatedRecoveryThread, TestPrepRecovery, TestTolerantUpdateProcessorRandomCloud, TriLevelCompositeIdRoutingTest, ZkShardTermsTest, CollectionReloadTest, CollectionsAPIAsyncDistributedZkTest, TestHdfsCloudBackupRestore, ComputePlanActionTest, NodeLostTriggerTest, SearchRateTriggerTest, SystemLogListenerTest, TriggerIntegrationTest]
[junit4] Completed [617/771 (2!)] on J1 in 197.89s, 14 tests, 1 failure <<< FAILURES!
[...truncated 48697 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 340 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/340/
3 tests failed.
FAILED: org.apache.solr.cloud.TestUtilizeNode.test
Error Message:
no replica should be present in 127.0.0.1:46579_solr
Stack Trace:
java.lang.AssertionError: no replica should be present in 127.0.0.1:46579_solr
at __randomizedtesting.SeedInfo.seed([9E217D9B085CD302:16754241A6A0BEFA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.TestUtilizeNode.test(TestUtilizeNode.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey
Error Message:
Could not load collection from ZK: collection1
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: collection1
at __randomizedtesting.SeedInfo.seed([9E217D9B085CD302:1506AE4A495A7886]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
at org.apache.solr.common.cloud.ZkStateReader.forceUpdateCollection(ZkStateReader.java:364)
at org.apache.solr.cloud.api.collections.ShardSplitTest.testSplitWithChaosMonkey(ShardSplitTest.java:396)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/collection1/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:130)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:54)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1215)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
... 44 more
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([9E217D9B085CD302:B84BC3A213537C6E]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTrigger(TriggerIntegrationTest.java:456)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12677 lines...]
[junit4] Suite: org.apache.solr.cloud.api.collections.ShardSplitTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/init-core-data-001
[junit4] 2> 1770606 WARN (SUITE-ShardSplitTest-seed#[9E217D9B085CD302]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=15 numCloses=15
[junit4] 2> 1770606 INFO (SUITE-ShardSplitTest-seed#[9E217D9B085CD302]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1770619 INFO (SUITE-ShardSplitTest-seed#[9E217D9B085CD302]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 1770619 INFO (SUITE-ShardSplitTest-seed#[9E217D9B085CD302]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1770619 INFO (SUITE-ShardSplitTest-seed#[9E217D9B085CD302]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 1770621 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1770621 INFO (Thread-4099) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1770621 INFO (Thread-4099) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1770682 ERROR (Thread-4099) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1770732 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.ZkTestServer start zk server on port:36273
[junit4] 2> 1770734 INFO (zkConnectionManagerCallback-1153-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1770939 INFO (zkConnectionManagerCallback-1155-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1770945 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1770946 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1770963 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1770964 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1770965 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1770965 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1770966 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1770966 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1770967 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1770980 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1770981 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1770982 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 1772507 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1772527 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1772527 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1772527 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1772527 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@43b63574{/,null,AVAILABLE}
[junit4] 2> 1772529 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@379d6c7e{HTTP/1.1,[http/1.1]}{127.0.0.1:44777}
[junit4] 2> 1772529 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.Server Started @1782478ms
[junit4] 2> 1772529 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=44777, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/control-001/cores}
[junit4] 2> 1772529 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1772529 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 1772529 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1772529 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1772529 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T12:46:18.598Z
[junit4] 2> 1772544 INFO (zkConnectionManagerCallback-1157-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1772546 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1772546 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/control-001/solr.xml
[junit4] 2> 1772579 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1772579 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1772580 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1772598 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36273/solr
[junit4] 2> 1772617 INFO (zkConnectionManagerCallback-1161-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1772630 INFO (zkConnectionManagerCallback-1163-thread-1-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1773323 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:44777_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1773324 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:44777_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44777_
[junit4] 2> 1773325 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:44777_ ] o.a.s.c.Overseer Overseer (id=72167976891842564-127.0.0.1:44777_-n_0000000000) starting
[junit4] 2> 1773430 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:44777_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44777_
[junit4] 2> 1773459 INFO (OverseerStateUpdate-72167976891842564-127.0.0.1:44777_-n_0000000000) [n:127.0.0.1:44777_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1773644 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:44777_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1773680 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:44777_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1773680 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:44777_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1773694 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:44777_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/control-001/cores
[junit4] 2> 1773766 INFO (zkConnectionManagerCallback-1169-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1773771 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1773772 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36273/solr ready
[junit4] 2> 1773773 INFO (qtp165025516-9131) [n:127.0.0.1:44777_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:44777_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1773811 INFO (OverseerThreadFactory-2811-thread-1-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 1773930 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1773930 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1774082 INFO (zkCallback-1162-thread-1-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1775009 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1775225 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 1775809 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1775886 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 1775886 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1775886 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1775887 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 1775889 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=13.8974609375, floorSegmentMB=1.619140625, forceMergeDeletesPctAllowed=23.39014344953578, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7424548770104626
[junit4] 2> 1776007 WARN (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1776250 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1776250 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1776251 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1776251 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1776252 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=40.4443359375, floorSegmentMB=1.6611328125, forceMergeDeletesPctAllowed=8.38784557714941, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1776253 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@64c427bd[control_collection_shard1_replica_n1] main]
[junit4] 2> 1776266 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1776267 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1776268 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1776269 INFO (searcherExecutor-2814-thread-1-processing-n:127.0.0.1:44777_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@64c427bd[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1776269 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590930945960050688
[junit4] 2> 1776303 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1776303 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1776303 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44777/control_collection_shard1_replica_n1/
[junit4] 2> 1776303 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1776303 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:44777/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 1776303 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1776305 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44777/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 1776339 INFO (zkCallback-1162-thread-2-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1776367 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1776368 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2438
[junit4] 2> 1776397 INFO (qtp165025516-9131) [n:127.0.0.1:44777_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1776505 INFO (zkCallback-1162-thread-2-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1777400 INFO (qtp165025516-9131) [n:127.0.0.1:44777_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:44777_&wt=javabin&version=2} status=0 QTime=3627
[junit4] 2> 1777518 INFO (zkConnectionManagerCallback-1173-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1777575 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1777575 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36273/solr ready
[junit4] 2> 1777576 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1777587 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1777725 INFO (OverseerThreadFactory-2811-thread-2-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 1777761 INFO (OverseerCollectionConfigSetProcessor-72167976891842564-127.0.0.1:44777_-n_0000000000) [n:127.0.0.1:44777_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1777777 WARN (OverseerThreadFactory-2811-thread-2-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 1778203 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1778203 INFO (qtp165025516-9135) [n:127.0.0.1:44777_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=615
[junit4] 2> 1778853 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-1-001 of type TLOG
[junit4] 2> 1778854 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1778875 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1778875 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1778875 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1778875 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@112af219{/,null,AVAILABLE}
[junit4] 2> 1778888 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2ca5f5a9{HTTP/1.1,[http/1.1]}{127.0.0.1:33317}
[junit4] 2> 1778888 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.Server Started @1788837ms
[junit4] 2> 1778888 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/tempDir-001/jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=33317, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-1-001/cores}
[junit4] 2> 1778888 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1778888 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 1778888 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1778888 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1778888 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T12:46:24.957Z
[junit4] 2> 1779094 INFO (zkConnectionManagerCallback-1175-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1779107 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1779107 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-1-001/solr.xml
[junit4] 2> 1779143 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1779143 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1779144 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1779179 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36273/solr
[junit4] 2> 1779220 INFO (zkConnectionManagerCallback-1179-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1779342 INFO (zkConnectionManagerCallback-1181-thread-1-processing-n:127.0.0.1:33317_) [n:127.0.0.1:33317_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1779398 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:33317_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1779400 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:33317_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1779401 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:33317_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33317_
[junit4] 2> 1779426 INFO (zkCallback-1172-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1779463 INFO (zkCallback-1180-thread-1-processing-n:127.0.0.1:33317_) [n:127.0.0.1:33317_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1779463 INFO (zkCallback-1162-thread-2-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1779799 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:33317_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1779852 INFO (OverseerCollectionConfigSetProcessor-72167976891842564-127.0.0.1:44777_-n_0000000000) [n:127.0.0.1:44777_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1779925 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:33317_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1779926 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:33317_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1779927 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:33317_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-1-001/cores
[junit4] 2> 1780127 INFO (qtp108459595-9180) [n:127.0.0.1:33317_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:33317_&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1780129 INFO (OverseerThreadFactory-2811-thread-3-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:33317_ for creating new replica
[junit4] 2> 1780144 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 1780144 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1781214 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1781252 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema [collection1_shard2_replica_t41] Schema name=test
[junit4] 2> 1781810 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1781867 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t41' using configuration from collection collection1, trusted=true
[junit4] 2> 1781868 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t41' (registry 'solr.core.collection1.shard2.replica_t41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1781868 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1781868 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [[collection1_shard2_replica_t41] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-1-001/cores/collection1_shard2_replica_t41], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-1-001/cores/collection1_shard2_replica_t41/data/]
[junit4] 2> 1781919 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=13.8974609375, floorSegmentMB=1.619140625, forceMergeDeletesPctAllowed=23.39014344953578, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7424548770104626
[junit4] 2> 1781968 WARN (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1782295 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1782295 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1782296 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1782296 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1782297 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=40.4443359375, floorSegmentMB=1.6611328125, forceMergeDeletesPctAllowed=8.38784557714941, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1782297 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.SolrIndexSearcher Opening [Searcher@22fe2396[collection1_shard2_replica_t41] main]
[junit4] 2> 1782298 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1782298 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1782334 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1782335 INFO (searcherExecutor-2825-thread-1-processing-n:127.0.0.1:33317_ x:collection1_shard2_replica_t41 s:shard2 c:collection1) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [collection1_shard2_replica_t41] Registered new searcher Searcher@22fe2396[collection1_shard2_replica_t41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1782347 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590930952333295616
[junit4] 2> 1782401 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1782401 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1782401 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33317/collection1_shard2_replica_t41/
[junit4] 2> 1782401 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1782401 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy http://127.0.0.1:33317/collection1_shard2_replica_t41/ has no replicas
[junit4] 2> 1782401 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1782401 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ZkController collection1_shard2_replica_t41 stopping background replication from leader
[junit4] 2> 1782402 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33317/collection1_shard2_replica_t41/ shard2
[junit4] 2> 1782509 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1782511 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2366
[junit4] 2> 1782567 INFO (qtp108459595-9180) [n:127.0.0.1:33317_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:33317_&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=2439
[junit4] 2> 1783501 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-2-001 of type TLOG
[junit4] 2> 1783502 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1783552 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1783552 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1783552 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1783553 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@270b9491{/,null,AVAILABLE}
[junit4] 2> 1783553 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1219064d{HTTP/1.1,[http/1.1]}{127.0.0.1:39333}
[junit4] 2> 1783553 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.Server Started @1793502ms
[junit4] 2> 1783553 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/tempDir-001/jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=39333, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-2-001/cores}
[junit4] 2> 1783553 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1783554 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 1783554 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1783554 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1783554 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T12:46:29.623Z
[junit4] 2> 1783640 INFO (zkConnectionManagerCallback-1185-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1783656 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1783656 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-2-001/solr.xml
[junit4] 2> 1783659 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1783659 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1783709 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1783731 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36273/solr
[junit4] 2> 1783765 INFO (zkConnectionManagerCallback-1189-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1783785 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100646482cc000b, likely client has closed socket
[junit4] 2> 1783822 INFO (zkConnectionManagerCallback-1191-thread-1-processing-n:127.0.0.1:39333_) [n:127.0.0.1:39333_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1783841 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:39333_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1783842 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:39333_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1783856 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:39333_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39333_
[junit4] 2> 1783857 INFO (zkCallback-1180-thread-2-processing-n:127.0.0.1:33317_) [n:127.0.0.1:33317_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1783857 INFO (zkCallback-1172-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1783940 INFO (zkCallback-1162-thread-1-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1783940 INFO (zkCallback-1190-thread-1-processing-n:127.0.0.1:39333_) [n:127.0.0.1:39333_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1784089 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:39333_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1784130 INFO (OverseerCollectionConfigSetProcessor-72167976891842564-127.0.0.1:44777_-n_0000000000) [n:127.0.0.1:44777_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1784160 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:39333_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1784160 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:39333_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1784161 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:39333_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-2-001/cores
[junit4] 2> 1784321 INFO (qtp108459595-9182) [n:127.0.0.1:33317_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:39333_&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1784323 INFO (OverseerThreadFactory-2811-thread-4-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:39333_ for creating new replica
[junit4] 2> 1784389 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 1784390 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1785414 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1785541 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema [collection1_shard1_replica_t43] Schema name=test
[junit4] 2> 1785906 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1785983 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t43' using configuration from collection collection1, trusted=true
[junit4] 2> 1785983 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t43' (registry 'solr.core.collection1.shard1.replica_t43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1785983 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1785984 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [[collection1_shard1_replica_t43] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-2-001/cores/collection1_shard1_replica_t43], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-2-001/cores/collection1_shard1_replica_t43/data/]
[junit4] 2> 1786003 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=13.8974609375, floorSegmentMB=1.619140625, forceMergeDeletesPctAllowed=23.39014344953578, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7424548770104626
[junit4] 2> 1786185 WARN (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1786291 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1786291 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1786291 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1786291 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1786292 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=40.4443359375, floorSegmentMB=1.6611328125, forceMergeDeletesPctAllowed=8.38784557714941, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1786293 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.SolrIndexSearcher Opening [Searcher@7f7df982[collection1_shard1_replica_t43] main]
[junit4] 2> 1786294 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1786294 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1786294 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1786295 INFO (searcherExecutor-2836-thread-1-processing-n:127.0.0.1:39333_ x:collection1_shard1_replica_t43 s:shard1 c:collection1) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [collection1_shard1_replica_t43] Registered new searcher Searcher@7f7df982[collection1_shard1_replica_t43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1786295 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590930956473073664
[junit4] 2> 1786312 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1786312 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1786312 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39333/collection1_shard1_replica_t43/
[junit4] 2> 1786312 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1786312 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy http://127.0.0.1:39333/collection1_shard1_replica_t43/ has no replicas
[junit4] 2> 1786312 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1786312 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ZkController collection1_shard1_replica_t43 stopping background replication from leader
[junit4] 2> 1786314 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39333/collection1_shard1_replica_t43/ shard1
[junit4] 2> 1786465 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1786466 INFO (qtp217719004-9212) [n:127.0.0.1:39333_ c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2076
[junit4] 2> 1786467 INFO (qtp108459595-9182) [n:127.0.0.1:33317_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:39333_&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=2146
[junit4] 2> 1786735 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-3-001 of type TLOG
[junit4] 2> 1786735 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T05:27:37+08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1786736 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1786736 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1786736 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1786736 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6c1fb5cd{/,null,AVAILABLE}
[junit4] 2> 1786737 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2032440a{HTTP/1.1,[http/1.1]}{127.0.0.1:46703}
[junit4] 2> 1786737 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.e.j.s.Server Started @1796685ms
[junit4] 2> 1786737 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/tempDir-001/jetty3, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/, hostPort=46703, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-3-001/cores}
[junit4] 2> 1786737 ERROR (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1786737 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 1786737 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1786737 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1786737 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T12:46:32.806Z
[junit4] 2> 1786739 INFO (zkConnectionManagerCallback-1195-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1786768 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1786768 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-3-001/solr.xml
[junit4] 2> 1786771 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1786771 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1786773 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1786775 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36273/solr
[junit4] 2> 1786777 INFO (zkConnectionManagerCallback-1199-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1786792 INFO (zkConnectionManagerCallback-1201-thread-1-processing-n:127.0.0.1:46703_) [n:127.0.0.1:46703_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1786796 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:46703_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1786797 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:46703_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1786799 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:46703_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46703_
[junit4] 2> 1786799 INFO (zkCallback-1172-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1786799 INFO (zkCallback-1180-thread-2-processing-n:127.0.0.1:33317_) [n:127.0.0.1:33317_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1786799 INFO (zkCallback-1162-thread-1-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1786799 INFO (zkCallback-1190-thread-1-processing-n:127.0.0.1:39333_) [n:127.0.0.1:39333_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1786828 INFO (zkCallback-1200-thread-1-processing-n:127.0.0.1:46703_) [n:127.0.0.1:46703_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1786923 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:46703_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1786944 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:46703_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1786944 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:46703_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1786945 INFO (TEST-ShardSplitTest.testSplitAfterFailedSplit-seed#[9E217D9B085CD302]) [n:127.0.0.1:46703_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-3-001/cores
[junit4] 2> 1787111 INFO (qtp108459595-9184) [n:127.0.0.1:33317_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:46703_&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1787125 INFO (OverseerCollectionConfigSetProcessor-72167976891842564-127.0.0.1:44777_-n_0000000000) [n:127.0.0.1:44777_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1787125 INFO (OverseerThreadFactory-2811-thread-5-processing-n:127.0.0.1:44777_) [n:127.0.0.1:44777_ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:46703_ for creating new replica
[junit4] 2> 1787127 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 1787127 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1788146 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1788163 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.IndexSchema [collection1_shard2_replica_t45] Schema name=test
[junit4] 2> 1788524 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1788561 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t45' using configuration from collection collection1, trusted=true
[junit4] 2> 1788562 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t45' (registry 'solr.core.collection1.shard2.replica_t45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@46e45467
[junit4] 2> 1788576 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1788576 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrCore [[collection1_shard2_replica_t45] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-3-001/cores/collection1_shard2_replica_t45], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.ShardSplitTest_9E217D9B085CD302-001/shard-3-001/cores/collection1_shard2_replica_t45/data/]
[junit4] 2> 1788596 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=10, maxMergeAtOnceExplicit=16, maxMergedSegmentMB=13.8974609375, floorSegmentMB=1.619140625, forceMergeDeletesPctAllowed=23.39014344953578, segmentsPerTier=39.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7424548770104626
[junit4] 2> 1788710 WARN (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1788894 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1788894 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1788895 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1788895 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1788896 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=22, maxMergeAtOnceExplicit=20, maxMergedSegmentMB=40.4443359375, floorSegmentMB=1.6611328125, forceMergeDeletesPctAllowed=8.38784557714941, segmentsPerTier=25.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1788897 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.SolrIndexSearcher Opening [Searcher@53708dc7[collection1_shard2_replica_t45] main]
[junit4] 2> 1788897 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1788898 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1788910 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1788911 INFO (searcherExecutor-2847-thread-1-processing-n:127.0.0.1:46703_ x:collection1_shard2_replica_t45 s:shard2 c:collection1) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrCore [collection1_shard2_replica_t45] Registered new searcher Searcher@53708dc7[collection1_shard2_replica_t45] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1788911 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590930959216148480
[junit4] 2> 1788933 INFO (qtp2136056561-9243) [n:127.0.0.1:46703_ c:collection1 s:shard2 x:collect
[...truncated too long message...]
o.e.j.s.session Stopped scavenging
[junit4] 2> 3843371 INFO (jetty-closer-7743-thread-9) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3843371 INFO (jetty-closer-7743-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72168098120990743-127.0.0.1:40007_solr-n_0000000006) closing
[junit4] 2> 3843371 INFO (OverseerStateUpdate-72168098120990743-127.0.0.1:40007_solr-n_0000000006) [n:127.0.0.1:40007_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40007_solr
[junit4] 2> 3843372 DEBUG (jetty-closer-7743-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 3843372 WARN (OverseerAutoScalingTriggerThread-72168098120990743-127.0.0.1:40007_solr-n_0000000006) [n:127.0.0.1:40007_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3843378 INFO (jetty-closer-7743-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@619e96c2{/solr,null,UNAVAILABLE}
[junit4] 2> 3843378 INFO (jetty-closer-7743-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3844848 INFO (jetty-closer-7743-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7be2b0aa{/solr,null,UNAVAILABLE}
[junit4] 2> 3844848 INFO (jetty-closer-7743-thread-4) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3844862 INFO (jetty-closer-7743-thread-12) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@73e753f9{/solr,null,UNAVAILABLE}
[junit4] 2> 3844863 INFO (jetty-closer-7743-thread-12) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3844863 ERROR (SUITE-TriggerIntegrationTest-seed#[9E217D9B085CD302]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3844864 INFO (SUITE-TriggerIntegrationTest-seed#[9E217D9B085CD302]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46464 46464
[junit4] 2> 3845097 INFO (Thread-5778) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46464 46464
[junit4] 2> 3845098 WARN (Thread-5778) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 44 /solr/collections/collection1/terms/shard1
[junit4] 2> 23 /solr/clusterprops.json
[junit4] 2> 23 /solr/aliases.json
[junit4] 2> 22 /solr/security.json
[junit4] 2> 12 /solr/configs/conf
[junit4] 2> 10 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 135 /solr/collections/collection1/state.json
[junit4] 2> 41 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 23 /solr/clusterstate.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 3 /solr/overseer_elect/election/72168098120990773-127.0.0.1:40178_solr-n_0000000016
[junit4] 2> 3 /solr/collections/collection1/leader_elect/shard1/election/72168098120990773-core_node10-n_0000000004
[junit4] 2> 3 /solr/overseer_elect/election/72168098120990743-127.0.0.1:40007_solr-n_0000000006
[junit4] 2> 2 /solr/overseer_elect/election/72168098120990755-127.0.0.1:38266_solr-n_0000000010
[junit4] 2> 2 /solr/overseer_elect/election/72168098120990749-127.0.0.1:40064_solr-n_0000000008
[junit4] 2> 2 /solr/overseer_elect/election/72168098120990764-127.0.0.1:36004_solr-n_0000000013
[junit4] 2> 2 /solr/overseer_elect/election/72168098120990726-127.0.0.1:46327_solr-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 23 /solr/live_nodes
[junit4] 2> 23 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/queue
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_9E217D9B085CD302-001
[junit4] 2> Jan 29, 2018 1:20:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=Lucene50(blocksize=128), x_s=PostingsFormat(name=LuceneFixedGap)}, docValues:{}, maxPointsInLeafNode=1293, maxMBSortInHeap=6.346978583443493, sim=RandomSimilarity(queryNorm=false): {}, locale=sl-SI, timezone=Asia/Hebron
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=195894880,total=524812288
[junit4] 2> NOTE: All tests run in this JVM: [SolrCmdDistributorTest, TestTolerantUpdateProcessorCloud, LeaderElectionIntegrationTest, ResponseLogComponentTest, CoreMergeIndexesAdminHandlerTest, MultiTermTest, MoveReplicaHDFSTest, TestFieldCacheReopen, ZkFailoverTest, UniqFieldsUpdateProcessorFactoryTest, TestFaceting, TestXmlQParser, TestAtomicUpdateErrorCases, StatelessScriptUpdateProcessorFactoryTest, SolrJmxReporterTest, TestDFRSimilarityFactory, TestDistributedStatsComponentCardinality, TestCoreBackup, TestOrdValues, TestFiltering, TestSSLRandomization, LeaderElectionTest, TestUseDocValuesAsStored2, DeleteReplicaTest, HdfsBasicDistributedZkTest, ConvertedLegacyTest, DistanceFunctionTest, ConnectionReuseTest, TestSuggestSpellingConverter, TestLazyCores, TestScoreJoinQPNoScore, ZkStateWriterTest, SolrCLIZkUtilsTest, BigEndianAscendingWordDeserializerTest, TestCoreContainer, TestCorePropertiesReload, HighlighterMaxOffsetTest, ClusterStateUpdateTest, TestStressReorder, SolrGangliaReporterTest, TestBulkSchemaConcurrent, AnalysisAfterCoreReloadTest, TestCryptoKeys, TestJsonFacetRefinement, TestSha256AuthenticationProvider, OverseerCollectionConfigSetProcessorTest, SchemaApiFailureTest, TestSizeLimitedDistributedMap, DistributedDebugComponentTest, TestSolrCloudSnapshots, TestTestInjection, AddBlockUpdateTest, RegexBoostProcessorTest, TestFieldCacheVsDocValues, ComputePlanActionTest, StatsReloadRaceTest, TestCloudRecovery, QueryResultKeyTest, URLClassifyProcessorTest, TestSubQueryTransformerDistrib, TestUtils, TestSolr4Spatial2, TestJavabinTupleStreamParser, AliasIntegrationTest, TestSolrCloudWithKerberosAlt, AutoAddReplicasIntegrationTest, DeleteInactiveReplicaTest, BasicDistributedZk2Test, JvmMetricsTest, SolrMetricManagerTest, TestSQLHandlerNonCloud, EnumFieldTest, TestLMJelinekMercerSimilarityFactory, SearchHandlerTest, AutoscalingHistoryHandlerTest, DistributedFacetPivotSmallTest, CleanupOldIndexTest, NodeLostTriggerTest, TestFieldResource, TestAuthorizationFramework, TestPolicyCloud, ConnectionManagerTest, BasicAuthIntegrationTest, TestAnalyzeInfixSuggestions, HdfsLockFactoryTest, TestConfigSetsAPI, ReplicationFactorTest, TestDistributedMissingSort, NotRequiredUniqueKeyTest, TestPivotHelperCode, TestSimpleTextCodec, TestDocBasedVersionConstraints, TestAuthenticationFramework, ClassificationUpdateProcessorTest, CollectionReloadTest, TestGenericDistributedQueue, SpellPossibilityIteratorTest, ShowFileRequestHandlerTest, MinimalSchemaTest, TestSolrCoreProperties, CacheHeaderTest, HttpTriggerListenerTest, TestRawTransformer, CopyFieldTest, TestInitQParser, TestObjectReleaseTracker, TestValueSourceCache, SmileWriterTest, MigrateRouteKeyTest, SearchHandlerTest, TestManagedSchemaAPI, TestBadConfig, NoCacheHeaderTest, RulesTest, SSLMigrationTest, ResponseHeaderTest, TestImplicitCoreProperties, TestTolerantSearch, TestSchemaNameResource, TestQuerySenderListener, TestSearcherReuse, TestConfigSetImmutable, TestComplexPhraseQParserPlugin, ClusterStateTest, FileBasedSpellCheckerTest, DocValuesTest, LIRRollingUpdatesTest, ChaosMonkeySafeLeaderTest, OverseerTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, ZkCLITest, TestDistributedGrouping, TestRecovery, TestRealTimeGet, TestReload, SimpleFacetsTest, StatsComponentTest, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, SuggesterFSTTest, CoreAdminHandlerTest, SuggesterTSTTest, TestTrie, SpatialFilterTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, TestUpdate, TestWordDelimiterFilterFactory, DirectUpdateHandlerOptimizeTest, DocValuesMultiTest, TestSolrDeletionPolicy1, XsltUpdateRequestHandlerTest, IndexBasedSpellCheckerTest, LukeRequestHandlerTest, TestOmitPositions, TermVectorComponentTest, MoreLikeThisHandlerTest, FieldAnalysisRequestHandlerTest, LoggingHandlerTest, IndexSchemaTest, ReturnFieldsTest, TestCSVResponseWriter, JsonLoaderTest, TestPHPSerializedResponseWriter, TestComponentsName, TestLFUCache, SolrTestCaseJ4Test, TestHighlightDedupGrouping, TestLuceneIndexBackCompat, CloudExitableDirectoryReaderTest, CollectionStateFormat2Test, CollectionsAPISolrJTest, DistribJoinFromCollectionTest, DocValuesNotIndexedTest, ForceLeaderTest, TestConfigSetsAPIZkFailure, TestDownShardTolerantSearch, TestLeaderElectionWithEmptyReplica, TestRandomRequestDistribution, TestRebalanceLeaders, TestStressLiveNodes, TestTolerantUpdateProcessorRandomCloud, CollectionsAPIAsyncDistributedZkTest, HdfsCollectionsAPIDistributedZkTest, SimpleCollectionCreateDeleteTest, TestCollectionAPI, TestLocalFSCloudBackupRestore, TestReplicaProperties, TriggerIntegrationTest]
[junit4] Completed [619/771 (3!)] on J2 in 225.72s, 14 tests, 1 failure <<< FAILURES!
[...truncated 48726 lines...]
Re: [JENKINS] Lucene-Solr-Tests-7.x - Build # 339 - Still Unstable
Posted by Gus Heck <gu...@gmail.com>.
Ah yes for some reason my IDE auto complete seems to like that one better.
I've corrected it several times over the last few weeks, but I guess I
missed that one. Thanks.
-Gus
On Sun, Jan 28, 2018 at 10:45 PM, David Smiley <da...@gmail.com>
wrote:
> RE failure of AliasIntegrationTest I think I see the problem. Gus, note
> that you called processAsync(...) instead of process(...) in tearDown when
> deleting the aliases. Perhaps it's too easy to call the async version :-)
> I'll fix.
>
>
> On Sun, Jan 28, 2018 at 9:17 PM Apache Jenkins Server <
> jenkins@builds.apache.org> wrote:
>
>> Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/339/
>>
>> 3 tests failed.
>> FAILED: org.apache.solr.cloud.AliasIntegrationTest.testMetadata
>>
>> Error Message:
>> Error from server at http://127.0.0.1:37832/solr: Collection :
>> collection2meta is part of alias meta3 remove or modify the alias before
>> removing this collection.
>>
>> Stack Trace:
>> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
>> Error from server at http://127.0.0.1:37832/solr: Collection :
>> collection2meta is part of alias meta3 remove or modify the alias before
>> removing this collection.
>> at __randomizedtesting.SeedInfo.seed([DA442618305C7D2C:
>> D3546C1C77B52145]:0)
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.
>> executeMethod(HttpSolrClient.java:643)
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(
>> HttpSolrClient.java:255)
>> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(
>> HttpSolrClient.java:244)
>> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.
>> doRequest(LBHttpSolrClient.java:483)
>> at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(
>> LBHttpSolrClient.java:413)
>> at org.apache.solr.client.solrj.impl.CloudSolrClient.
>> sendRequest(CloudSolrClient.java:1104)
>> at org.apache.solr.client.solrj.impl.CloudSolrClient.
>> requestWithRetryOnStaleState(CloudSolrClient.java:884)
>> at org.apache.solr.client.solrj.impl.CloudSolrClient.request(
>> CloudSolrClient.java:817)
>> at org.apache.solr.client.solrj.SolrRequest.process(
>> SolrRequest.java:194)
>> at org.apache.solr.client.solrj.SolrRequest.process(
>> SolrRequest.java:211)
>> at org.apache.solr.cloud.AliasIntegrationTest.tearDown(
>> AliasIntegrationTest.java:95)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(
>> RandomizedRunner.java:1737)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
>> at com.carrotsearch.randomizedtesting.rules.
>> SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.
>> java:57)
>> at org.apache.lucene.util.TestRuleSetupTeardownChained$
>> 1.evaluate(TestRuleSetupTeardownChained.java:49)
>> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(
>> AbstractBeforeAfterRule.java:45)
>> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(
>> TestRuleThreadAndTestName.java:48)
>> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures
>> $1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> at org.apache.lucene.util.TestRuleMarkFailure$1.
>> evaluate(TestRuleMarkFailure.java:47)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.ThreadLeakControl$
>> StatementRunner.run(ThreadLeakControl.java:368)
>> at com.carrotsearch.randomizedtesting.ThreadLeakControl.
>> forkTimeoutingTask(ThreadLeakControl.java:817)
>> at com.carrotsearch.randomizedtesting.
>> ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
>> at com.carrotsearch.randomizedtesting.RandomizedRunner.
>> runSingleTest(RandomizedRunner.java:943)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.rules.
>> SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.
>> java:57)
>> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(
>> AbstractBeforeAfterRule.java:45)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(
>> TestRuleStoreClassName.java:41)
>> at com.carrotsearch.randomizedtesting.rules.
>> NoShadowingOrOverridesOnMethodsRule$1.evaluate(
>> NoShadowingOrOverridesOnMethodsRule.java:40)
>> at com.carrotsearch.randomizedtesting.rules.
>> NoShadowingOrOverridesOnMethodsRule$1.evaluate(
>> NoShadowingOrOverridesOnMethodsRule.java:40)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(
>> TestRuleAssertionsRequired.java:53)
>> at org.apache.lucene.util.TestRuleMarkFailure$1.
>> evaluate(TestRuleMarkFailure.java:47)
>> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures
>> $1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(
>> TestRuleIgnoreTestSuites.java:54)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.ThreadLeakControl$
>> StatementRunner.run(ThreadLeakControl.java:368)
>> at java.lang.Thread.run(Thread.java:748)
>>
>>
>> FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.
>> testEventFromRestoredState
>>
>> Error Message:
>>
>>
>> Stack Trace:
>> java.lang.AssertionError
>> at __randomizedtesting.SeedInfo.seed([DA442618305C7D2C:
>> DA7292D81951D946]:0)
>> at org.junit.Assert.fail(Assert.java:92)
>> at org.junit.Assert.assertTrue(Assert.java:43)
>> at org.junit.Assert.assertTrue(Assert.java:54)
>> at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.
>> testEventFromRestoredState(TriggerIntegrationTest.java:760)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(
>> RandomizedRunner.java:1737)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
>> at com.carrotsearch.randomizedtesting.rules.
>> SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.
>> java:57)
>> at org.apache.lucene.util.TestRuleSetupTeardownChained$
>> 1.evaluate(TestRuleSetupTeardownChained.java:49)
>> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(
>> AbstractBeforeAfterRule.java:45)
>> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(
>> TestRuleThreadAndTestName.java:48)
>> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures
>> $1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> at org.apache.lucene.util.TestRuleMarkFailure$1.
>> evaluate(TestRuleMarkFailure.java:47)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.ThreadLeakControl$
>> StatementRunner.run(ThreadLeakControl.java:368)
>> at com.carrotsearch.randomizedtesting.ThreadLeakControl.
>> forkTimeoutingTask(ThreadLeakControl.java:817)
>> at com.carrotsearch.randomizedtesting.
>> ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
>> at com.carrotsearch.randomizedtesting.RandomizedRunner.
>> runSingleTest(RandomizedRunner.java:943)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.rules.
>> SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.
>> java:57)
>> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(
>> AbstractBeforeAfterRule.java:45)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(
>> TestRuleStoreClassName.java:41)
>> at com.carrotsearch.randomizedtesting.rules.
>> NoShadowingOrOverridesOnMethodsRule$1.evaluate(
>> NoShadowingOrOverridesOnMethodsRule.java:40)
>> at com.carrotsearch.randomizedtesting.rules.
>> NoShadowingOrOverridesOnMethodsRule$1.evaluate(
>> NoShadowingOrOverridesOnMethodsRule.java:40)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(
>> TestRuleAssertionsRequired.java:53)
>> at org.apache.lucene.util.TestRuleMarkFailure$1.
>> evaluate(TestRuleMarkFailure.java:47)
>> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures
>> $1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(
>> TestRuleIgnoreTestSuites.java:54)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.ThreadLeakControl$
>> StatementRunner.run(ThreadLeakControl.java:368)
>> at java.lang.Thread.run(Thread.java:748)
>>
>>
>> FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.
>> testMetricTrigger
>>
>> Error Message:
>>
>>
>> Stack Trace:
>> java.lang.AssertionError
>> at __randomizedtesting.SeedInfo.seed([DA442618305C7D2C:
>> 604811976FB4AB63]:0)
>> at org.junit.Assert.fail(Assert.java:92)
>> at org.junit.Assert.assertTrue(Assert.java:43)
>> at org.junit.Assert.assertNull(Assert.java:551)
>> at org.junit.Assert.assertNull(Assert.java:562)
>> at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.
>> testMetricTrigger(TriggerIntegrationTest.java:1575)
>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>> at sun.reflect.NativeMethodAccessorImpl.invoke(
>> NativeMethodAccessorImpl.java:62)
>> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
>> DelegatingMethodAccessorImpl.java:43)
>> at java.lang.reflect.Method.invoke(Method.java:498)
>> at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(
>> RandomizedRunner.java:1737)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
>> at com.carrotsearch.randomizedtesting.rules.
>> SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.
>> java:57)
>> at org.apache.lucene.util.TestRuleSetupTeardownChained$
>> 1.evaluate(TestRuleSetupTeardownChained.java:49)
>> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(
>> AbstractBeforeAfterRule.java:45)
>> at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(
>> TestRuleThreadAndTestName.java:48)
>> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures
>> $1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> at org.apache.lucene.util.TestRuleMarkFailure$1.
>> evaluate(TestRuleMarkFailure.java:47)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.ThreadLeakControl$
>> StatementRunner.run(ThreadLeakControl.java:368)
>> at com.carrotsearch.randomizedtesting.ThreadLeakControl.
>> forkTimeoutingTask(ThreadLeakControl.java:817)
>> at com.carrotsearch.randomizedtesting.
>> ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
>> at com.carrotsearch.randomizedtesting.RandomizedRunner.
>> runSingleTest(RandomizedRunner.java:943)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
>> at com.carrotsearch.randomizedtesting.
>> RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.rules.
>> SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.
>> java:57)
>> at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(
>> AbstractBeforeAfterRule.java:45)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(
>> TestRuleStoreClassName.java:41)
>> at com.carrotsearch.randomizedtesting.rules.
>> NoShadowingOrOverridesOnMethodsRule$1.evaluate(
>> NoShadowingOrOverridesOnMethodsRule.java:40)
>> at com.carrotsearch.randomizedtesting.rules.
>> NoShadowingOrOverridesOnMethodsRule$1.evaluate(
>> NoShadowingOrOverridesOnMethodsRule.java:40)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(
>> TestRuleAssertionsRequired.java:53)
>> at org.apache.lucene.util.TestRuleMarkFailure$1.
>> evaluate(TestRuleMarkFailure.java:47)
>> at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures
>> $1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
>> at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(
>> TestRuleIgnoreTestSuites.java:54)
>> at com.carrotsearch.randomizedtesting.rules.
>> StatementAdapter.evaluate(StatementAdapter.java:36)
>> at com.carrotsearch.randomizedtesting.ThreadLeakControl$
>> StatementRunner.run(ThreadLeakControl.java:368)
>> at java.lang.Thread.run(Thread.java:748)
>>
>>
>>
>>
>> Build Log:
>> [...truncated 12951 lines...]
>> [junit4] Suite: org.apache.solr.cloud.autoscaling.
>> TriggerIntegrationTest
>> [junit4] 2> 2357318 INFO (SUITE-TriggerIntegrationTest-
>> seed#[DA442618305C7D2C]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom
>> sanity checks: test.solr.allowed.securerandom=null &
>> java.security.egd=file:/dev/./urandom
>> [junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/
>> workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/
>> J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_
>> DA442618305C7D2C-001/init-core-data-001
>> [junit4] 2> 2357319 INFO (SUITE-TriggerIntegrationTest-
>> seed#[DA442618305C7D2C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using
>> PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=
>> true
>> [junit4] 2> 2357319 INFO (SUITE-TriggerIntegrationTest-
>> seed#[DA442618305C7D2C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized
>> ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=,
>> ssl=NaN, value=NaN, clientAuth=NaN)
>> [junit4] 2> 2357320 INFO (SUITE-TriggerIntegrationTest-
>> seed#[DA442618305C7D2C]-worker) [ ] o.a.s.c.MiniSolrCloudCluster
>> Starting cluster of 2 servers in /home/jenkins/jenkins-slave/
>> workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/
>> J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_
>> DA442618305C7D2C-001/tempDir-001
>> [junit4] 2> 2357320 INFO (SUITE-TriggerIntegrationTest-
>> seed#[DA442618305C7D2C]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK
>> TEST SERVER
>> [junit4] 2> 2357320 INFO (Thread-4693) [ ] o.a.s.c.ZkTestServer
>> client port:0.0.0.0/0.0.0.0:0
>> [junit4] 2> 2357320 INFO (Thread-4693) [ ] o.a.s.c.ZkTestServer
>> Starting server
>> [junit4] 2> 2357322 ERROR (Thread-4693) [ ]
>> o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper
>> server won't take any action on ERROR or SHUTDOWN server state changes
>> [junit4] 2> 2357421 INFO (SUITE-TriggerIntegrationTest-
>> seed#[DA442618305C7D2C]-worker) [ ] o.a.s.c.ZkTestServer start zk
>> server on port:46316
>> [junit4] 2> 2357422 INFO (zkConnectionManagerCallback-2655-thread-1)
>> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357425 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp:
>> 2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a
>> 2a2615f0a8
>> [junit4] 2> 2357425 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp:
>> 2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a
>> 2a2615f0a8
>> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.e.j.s.session DefaultSessionIdManager workerName=node0
>> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.e.j.s.session DefaultSessionIdManager workerName=node0
>> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.e.j.s.session No SessionScavenger set, using defaults
>> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.e.j.s.session No SessionScavenger set, using defaults
>> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.e.j.s.session Scavenging every 600000ms
>> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.e.j.s.session Scavenging every 600000ms
>> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@
>> 676003ea{/solr,null,AVAILABLE}
>> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@
>> 275f23bf{/solr,null,AVAILABLE}
>> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.e.j.s.AbstractConnector Started ServerConnector@655c52be{SSL,[ssl,
>> http/1.1]}{127.0.0.1:35199}
>> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.e.j.s.AbstractConnector Started ServerConnector@190ae2b1{SSL,[ssl,
>> http/1.1]}{127.0.0.1:35307}
>> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.e.j.s.Server Started @2368674ms
>> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.e.j.s.Server Started @2368674ms
>> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
>> hostPort=35199}
>> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
>> hostPort=35307}
>> [junit4] 2> 2357427 ERROR (jetty-launcher-2652-thread-1) [ ]
>> o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may
>> be missing or incomplete.
>> [junit4] 2> 2357427 ERROR (jetty-launcher-2652-thread-2) [ ]
>> o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may
>> be missing or incomplete.
>> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr?
>> version 7.3.0
>> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr?
>> version 7.3.0
>> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port
>> null
>> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port
>> null
>> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
>> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
>> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
>> 2018-01-29T01:06:00.681Z
>> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
>> 2018-01-29T01:06:00.681Z
>> [junit4] 2> 2357435 INFO (zkConnectionManagerCallback-2658-thread-1)
>> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357436 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
>> [junit4] 2> 2357440 INFO (zkConnectionManagerCallback-2659-thread-1)
>> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357441 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
>> [junit4] 2> 2357445 INFO (jetty-launcher-2652-thread-1) [ ]
>> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
>> [junit4] 2> 2357445 INFO (jetty-launcher-2652-thread-2) [ ]
>> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
>> [junit4] 2> 2357446 INFO (zkConnectionManagerCallback-2663-thread-1)
>> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357452 INFO (zkConnectionManagerCallback-2667-thread-1)
>> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357453 INFO (zkConnectionManagerCallback-
>> 2669-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357453 INFO (zkConnectionManagerCallback-
>> 2671-thread-1-processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr
>> ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357519 INFO (jetty-launcher-2652-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.Overseer Overseer (id=null) closing
>> [junit4] 2> 2357519 INFO (jetty-launcher-2652-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.OverseerElectionContext I am going
>> to be the leader 127.0.0.1:35199_solr
>> [junit4] 2> 2357520 INFO (jetty-launcher-2652-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.Overseer Overseer
>> (id=99430430184177669-127.0.0.1:35199_solr-n_0000000000) starting
>> [junit4] 2> 2357521 INFO (jetty-launcher-2652-thread-2)
>> [n:127.0.0.1:35307_solr ] o.a.s.c.Overseer Overseer (id=null) closing
>> [junit4] 2> 2357521 INFO (jetty-launcher-2652-thread-2)
>> [n:127.0.0.1:35307_solr ] o.a.s.c.ZkController Register node as live
>> in ZooKeeper:/live_nodes/127.0.0.1:35307_solr
>> [junit4] 2> 2357534 INFO (zkCallback-2670-thread-1-
>> processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
>> [junit4] 2> 2357535 INFO (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
>> [junit4] 2> 2357537 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Adding
>> .autoAddReplicas trigger
>> [junit4] 2> 2357538 INFO (jetty-launcher-2652-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.ZkController Register node as live
>> in ZooKeeper:/live_nodes/127.0.0.1:35199_solr
>> [junit4] 2> 2357538 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing
>> /autoscaling.json with znode version 1
>> [junit4] 2> 2357546 INFO (zkCallback-2670-thread-1-
>> processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
>> [junit4] 2> 2357546 INFO (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
>> [junit4] 2> 2357546 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Initial
>> livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr]
>> [junit4] 2> 2357546 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
>> znodeVersion 1, lastZnodeVersion -1
>> [junit4] 2> 2357546 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
>> trigger updates upto znodeVersion 1
>> [junit4] 2> 2357547 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean
>> old nodeAdded markers
>> [junit4] 2> 2357547 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
>> znodeVersion 1, lastZnodeVersion 1
>> [junit4] 2> 2357548 DEBUG (ScheduledTrigger-4813-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
>> NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
>> [junit4] 2> 2357581 INFO (jetty-launcher-2652-thread-2)
>> [n:127.0.0.1:35307_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring
>> for 'solr_35307.solr.node' (registry 'solr.node') enabled at server:
>> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
>> [junit4] 2> 2357582 INFO (jetty-launcher-2652-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring
>> for 'solr_35199.solr.node' (registry 'solr.node') enabled at server:
>> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
>> [junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-2)
>> [n:127.0.0.1:35307_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring
>> for 'solr_35307.solr.jvm' (registry 'solr.jvm') enabled at server:
>> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
>> [junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-2)
>> [n:127.0.0.1:35307_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring
>> for 'solr_35307.solr.jetty' (registry 'solr.jetty') enabled at server:
>> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
>> [junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring
>> for 'solr_35199.solr.jvm' (registry 'solr.jvm') enabled at server:
>> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
>> [junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring
>> for 'solr_35199.solr.jetty' (registry 'solr.jetty') enabled at server:
>> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
>> [junit4] 2> 2357590 INFO (jetty-launcher-2652-thread-2)
>> [n:127.0.0.1:35307_solr ] o.a.s.c.CorePropertiesLocator Found 0 core
>> definitions underneath /home/jenkins/jenkins-slave/
>> workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/
>> J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_
>> DA442618305C7D2C-001/tempDir-001/node2/.
>> [junit4] 2> 2357590 INFO (jetty-launcher-2652-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.CorePropertiesLocator Found 0 core
>> definitions underneath /home/jenkins/jenkins-slave/
>> workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/
>> J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_
>> DA442618305C7D2C-001/tempDir-001/node1/.
>> [junit4] 2> 2357622 INFO (zkConnectionManagerCallback-2677-thread-1)
>> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357624 INFO (zkConnectionManagerCallback-2681-thread-1)
>> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357624 INFO (SUITE-TriggerIntegrationTest-
>> seed#[DA442618305C7D2C]-worker) [ ] o.a.s.c.c.ZkStateReader Updated
>> live nodes from ZooKeeper... (0) -> (2)
>> [junit4] 2> 2357625 INFO (SUITE-TriggerIntegrationTest-
>> seed#[DA442618305C7D2C]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider
>> Cluster at 127.0.0.1:46316/solr ready
>> [junit4] 2> 2357637 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
>> [junit4] 2> 2357637 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.c.a.TriggerIntegrationTest
>> /autoscaling.json reset, new znode version 2
>> [junit4] 2> 2357637 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
>> version 2
>> [junit4] 2> 2357637 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
>> trigger updates upto znodeVersion 2
>> [junit4] 2> 2357638 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean
>> old nodeLost markers
>> [junit4] 2> 2357638 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean
>> old nodeAdded markers
>> [junit4] 2> 2357639 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
>> znodeVersion 2, lastZnodeVersion 2
>> [junit4] 2> 2357664 INFO (qtp1081540318-14383) [n:127.0.0.1:35199_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
>> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=
>> javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>> [junit4] 2> 2357665 INFO (qtp702010038-14390) [n:127.0.0.1:35307_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
>> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=
>> javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>> [junit4] 2> 2357666 DEBUG (qtp702010038-14382) [n:127.0.0.1:35307_solr
>> ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
>> [junit4] 2> 2357666 INFO (qtp702010038-14382) [n:127.0.0.1:35307_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling
>> params={wt=javabin&version=2} status=0 QTime=14
>> [junit4] 2> 2357666 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
>> version 3
>> [junit4] 2> 2357669 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr,
>> 127.0.0.1:35199_solr]
>> [junit4] 2> 2357669 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1
>> instantiated with properties: {event=nodeAdded, waitFor=0,
>> actions=[{name=test, class=org.apache.solr.cloud.autoscaling.
>> TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
>> [junit4] 2> 2357669 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
>> trigger updates upto znodeVersion 3
>> [junit4] 2> 2357669 INFO (qtp1081540318-14387) [n:127.0.0.1:35199_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
>> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=
>> javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>> [junit4] 2> 2357670 INFO (qtp702010038-14384) [n:127.0.0.1:35307_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
>> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=
>> javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>> [junit4] 2> 2357670 INFO (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest
>> TestTriggerAction init
>> [junit4] 2> 2357670 DEBUG (qtp702010038-14386) [n:127.0.0.1:35307_solr
>> ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
>> [junit4] 2> 2357671 INFO (qtp702010038-14386) [n:127.0.0.1:35307_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling
>> params={wt=javabin&version=2} status=0 QTime=2
>> [junit4] 2> 2357678 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
>> version 4
>> [junit4] 2> 2357678 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr,
>> 127.0.0.1:35199_solr]
>> [junit4] 2> 2357678 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2
>> instantiated with properties: {event=nodeAdded, waitFor=0,
>> actions=[{name=test, class=org.apache.solr.cloud.autoscaling.
>> TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
>> [junit4] 2> 2357678 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr,
>> 127.0.0.1:35199_solr]
>> [junit4] 2> 2357679 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1
>> instantiated with properties: {event=nodeAdded, waitFor=0,
>> actions=[{name=test, class=org.apache.solr.cloud.autoscaling.
>> TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
>> [junit4] 2> 2357679 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean
>> old nodeLost markers
>> [junit4] 2> 2357679 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
>> znodeVersion 4, lastZnodeVersion 3
>> [junit4] 2> 2357679 DEBUG (ScheduledTrigger-4813-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
>> NodeAddedTrigger node_added_trigger1
>> [junit4] 2> 2357679 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
>> trigger updates upto znodeVersion 4
>> [junit4] 2> 2357679 DEBUG (ScheduledTrigger-4813-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes:
>> 2
>> [junit4] 2> 2357680 INFO (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest
>> TestTriggerAction init
>> [junit4] 2> 2357681 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean
>> old nodeLost markers
>> [junit4] 2> 2357681 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
>> znodeVersion 4, lastZnodeVersion 4
>> [junit4] 2> 2357681 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
>> NodeAddedTrigger node_added_trigger2
>> [junit4] 2> 2357681 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes:
>> 2
>> [junit4] 2> 2357681 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server
>> jetty-9.4.8.v20171121, build timestamp: 2017-11-22T09:27:37+12:00, git
>> hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
>> [junit4] 2> 2357682 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session
>> DefaultSessionIdManager workerName=node0
>> [junit4] 2> 2357682 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session No
>> SessionScavenger set, using defaults
>> [junit4] 2> 2357682 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session
>> Scavenging every 600000ms
>> [junit4] 2> 2357682 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@
>> 46112453{/solr,null,AVAILABLE}
>> [junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.e.j.s.AbstractConnector Started ServerConnector@435c6f3d{SSL,[ssl,
>> http/1.1]}{127.0.0.1:45825}
>> [junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server
>> Started @2368930ms
>> [junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
>> hostPort=45825}
>> [junit4] 2> 2357683 ERROR (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may
>> be missing or incomplete.
>> [junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr?
>> version 7.3.0
>> [junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port
>> null
>> [junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
>> [junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
>> 2018-01-29T01:06:00.936Z
>> [junit4] 2> 2357684 INFO (zkConnectionManagerCallback-2683-thread-1)
>> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357685 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
>> [junit4] 2> 2357691 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
>> [junit4] 2> 2357692 INFO (zkConnectionManagerCallback-2687-thread-1)
>> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357693 INFO (zkConnectionManagerCallback-
>> 2689-thread-1-processing-n:127.0.0.1:45825_solr) [n:127.0.0.1:45825_solr
>> ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2357697 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr
>> ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
>> [junit4] 2> 2357698 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr
>> ] o.a.s.c.Overseer Overseer (id=null) closing
>> [junit4] 2> 2357699 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr
>> ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/
>> 127.0.0.1:45825_solr
>> [junit4] 2> 2357700 INFO (zkCallback-2670-thread-1-
>> processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
>> [junit4] 2> 2357700 INFO (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
>> [junit4] 2> 2357700 INFO (zkCallback-2688-thread-1-
>> processing-n:127.0.0.1:45825_solr) [n:127.0.0.1:45825_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
>> [junit4] 2> 2357700 INFO (zkCallback-2680-thread-1) [ ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
>> [junit4] 2> 2357740 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr
>> ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45825.solr.node'
>> (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.
>> JmxMBeanServer@6c7a445
>> [junit4] 2> 2357749 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr
>> ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45825.solr.jvm'
>> (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.
>> JmxMBeanServer@6c7a445
>> [junit4] 2> 2357749 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr
>> ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45825.solr.jetty'
>> (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.
>> JmxMBeanServer@6c7a445
>> [junit4] 2> 2357750 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr
>> ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath
>> /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.
>> x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.
>> TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001/node3/.
>> [junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
>> NodeAddedTrigger node_added_trigger1
>> [junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes:
>> 3
>> [junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new
>> node: 127.0.0.1:45825_solr at time 1642293927204014
>> [junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger
>> node_added_trigger1 firing registered processor for nodes: [127.0.0.1:45825_solr]
>> added at times [1642293927204014], now=1642293927367265
>> [junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++
>> Cooldown inactive - processing event: {
>> [junit4] 2> "id":"5d5a85a25d4aeT3me4vw7c8et15q9500dmer1ag",
>> [junit4] 2> "source":"node_added_trigger1",
>> [junit4] 2> "eventTime":1642293927204014,
>> [junit4] 2> "eventType":"NODEADDED",
>> [junit4] 2> "properties":{
>> [junit4] 2> "eventTimes":[1642293927204014],
>> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
>> [junit4] 2> 2358682 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all
>> triggers: [node_added_trigger2, node_added_trigger1]
>> [junit4] 2> 2358683 INFO (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection
>> .system does not exist, disabling logging.
>> [junit4] 2> 2358683 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
>> [junit4] 2> "id":"5d5a85a25d4aeT3me4vw7c8et15q9500dmer1ag",
>> [junit4] 2> "source":"node_added_trigger1",
>> [junit4] 2> "eventTime":1642293927204014,
>> [junit4] 2> "eventType":"NODEADDED",
>> [junit4] 2> "properties":{
>> [junit4] 2> "eventTimes":[1642293927204014],
>> [junit4] 2> "_enqueue_time_":1642293928447431,
>> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
>> [junit4] 2> 2358683 INFO (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.TriggerIntegrationTest action executed from
>> node_added_trigger1
>> [junit4] 2> 2358684 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after
>> 5000ms
>> [junit4] 2> 2358684 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after
>> 5000ms
>> [junit4] 2> 2358684 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=
>> 5d5a85a25d4aeT3me4vw7c8et15q9500dmer1ag
>> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
>> NodeAddedTrigger node_added_trigger2
>> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-4)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
>> NodeAddedTrigger node_added_trigger1
>> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes:
>> 3
>> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-4)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes:
>> 3
>> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new
>> node: 127.0.0.1:45825_solr at time 1642298930755018
>> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger
>> node_added_trigger2 firing registered processor for nodes: [127.0.0.1:45825_solr]
>> added at times [1642298930755018], now=1642298930823201
>> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++
>> Cooldown inactive - processing event: {
>> [junit4] 2> "id":"5d5a98461f5caT3me4vw7c8et15q9500dmer1ai",
>> [junit4] 2> "source":"node_added_trigger2",
>> [junit4] 2> "eventTime":1642298930755018,
>> [junit4] 2> "eventType":"NODEADDED",
>> [junit4] 2> "properties":{
>> [junit4] 2> "eventTimes":[1642298930755018],
>> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
>> [junit4] 2> 2363685 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all
>> triggers: [node_added_trigger2, node_added_trigger1]
>> [junit4] 2> 2363685 INFO (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection
>> .system does not exist, disabling logging.
>> [junit4] 2> 2363685 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
>> [junit4] 2> "id":"5d5a98461f5caT3me4vw7c8et15q9500dmer1ai",
>> [junit4] 2> "source":"node_added_trigger2",
>> [junit4] 2> "eventTime":1642298930755018,
>> [junit4] 2> "eventType":"NODEADDED",
>> [junit4] 2> "properties":{
>> [junit4] 2> "eventTimes":[1642298930755018],
>> [junit4] 2> "_enqueue_time_":1642298931021681,
>> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
>> [junit4] 2> 2363685 INFO (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.TriggerIntegrationTest last action at 1517187961936000000
>> current time = 1517187966938000000
>> [junit4] 2> real diff: 5002000000 <(500)%20200-0000>
>> [junit4] 2> min diff: 4998000000
>> [junit4] 2> 2363685 INFO (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.TriggerIntegrationTest action executed from
>> node_added_trigger2
>> [junit4] 2> 2363686 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after
>> 5000ms
>> [junit4] 2> 2363686 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after
>> 5000ms
>> [junit4] 2> 2363686 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers -- processing took 0 ms for event id=
>> 5d5a98461f5caT3me4vw7c8et15q9500dmer1ai
>> [junit4] 2> 2363689 INFO (qtp1081540318-14383) [n:127.0.0.1:35199_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
>> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=
>> javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>> [junit4] 2> 2363691 INFO (qtp702010038-14390) [n:127.0.0.1:35307_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
>> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=
>> javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>> [junit4] 2> 2363705 INFO (qtp2109945748-14443) [n:127.0.0.1:45825_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
>> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=
>> javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>> [junit4] 2> 2363705 DEBUG (qtp702010038-14382) [n:127.0.0.1:35307_solr
>> ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
>> [junit4] 2> 2363706 INFO (qtp702010038-14382) [n:127.0.0.1:35307_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling
>> params={wt=javabin&version=2} status=0 QTime=18
>> [junit4] 2> 2363706 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
>> version 5
>> [junit4] 2> 2363706 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr,
>> 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
>> [junit4] 2> 2363706 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2
>> instantiated with properties: {event=nodeAdded, waitFor=0,
>> actions=[{name=test, class=org.apache.solr.cloud.autoscaling.
>> TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
>> [junit4] 2> 2363706 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:35307_solr,
>> 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
>> [junit4] 2> 2363707 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr,
>> 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
>> [junit4] 2> 2363707 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1
>> instantiated with properties: {event=nodeAdded, waitFor=0,
>> actions=[{name=test, class=org.apache.solr.cloud.autoscaling.
>> TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
>> [junit4] 2> 2363707 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
>> trigger updates upto znodeVersion 5
>> [junit4] 2> 2363707 INFO (qtp1081540318-14387) [n:127.0.0.1:35199_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
>> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=
>> javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>> [junit4] 2> 2363707 INFO (qtp702010038-14384) [n:127.0.0.1:35307_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
>> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=
>> javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>> [junit4] 2> 2363708 INFO (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest
>> TestTriggerAction init
>> [junit4] 2> 2363708 INFO (qtp2109945748-14445) [n:127.0.0.1:45825_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
>> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=
>> javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
>> [junit4] 2> 2363708 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
>> znodeVersion 5, lastZnodeVersion 5
>> [junit4] 2> 2363708 DEBUG (qtp702010038-14386) [n:127.0.0.1:35307_solr
>> ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
>> [junit4] 2> 2363708 DEBUG (ScheduledTrigger-4813-thread-4)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
>> NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
>> [junit4] 2> 2363708 INFO (qtp702010038-14386) [n:127.0.0.1:35307_solr
>> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling
>> params={wt=javabin&version=2} status=0 QTime=2
>> [junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
>> version 6
>> [junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr,
>> 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
>> [junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2
>> instantiated with properties: {event=nodeAdded, waitFor=0,
>> actions=[{name=test, class=org.apache.solr.cloud.autoscaling.
>> TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
>> [junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:35307_solr,
>> 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
>> [junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr,
>> 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
>> [junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1
>> instantiated with properties: {event=nodeAdded, waitFor=0,
>> actions=[{name=test, class=org.apache.solr.cloud.autoscaling.
>> TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
>> [junit4] 2> 2363710 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:35307_solr,
>> 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
>> [junit4] 2> 2363710 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
>> trigger updates upto znodeVersion 6
>> [junit4] 2> 2363711 INFO (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest
>> TestTriggerAction init
>> [junit4] 2> 2363711 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
>> znodeVersion 6, lastZnodeVersion 6
>> [junit4] 2> 2363712 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.e.j.s.AbstractConnector Stopped ServerConnector@435c6f3d{SSL,[ssl,
>> http/1.1]}{127.0.0.1:0}
>> [junit4] 2> 2363712 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.c.CoreContainer Shutting down CoreContainer instance=1700547441
>> [junit4] 2> 2363712 DEBUG (ScheduledTrigger-4813-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
>> NodeLostTrigger: node_lost_trigger2 with currently live nodes: 3
>> [junit4] 2> 2363712 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node,
>> tag=null
>> [junit4] 2> 2363712 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.
>> reporters.SolrJmxReporter@4eb5a754: rootName = solr_45825, domain =
>> solr.node, service url = null, agent id = null] for registry solr.node /
>> com.codahale.metrics.MetricRegistry@3907f8dd
>> [junit4] 2> 2363717 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm,
>> tag=null
>> [junit4] 2> 2363717 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.
>> reporters.SolrJmxReporter@103306db: rootName = solr_45825, domain =
>> solr.jvm, service url = null, agent id = null] for registry solr.jvm /
>> com.codahale.metrics.MetricRegistry@dc0593e
>> [junit4] 2> 2363721 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty,
>> tag=null
>> [junit4] 2> 2363721 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.
>> reporters.SolrJmxReporter@23607bff: rootName = solr_45825, domain =
>> solr.jetty, service url = null, agent id = null] for registry solr.jetty /
>> com.codahale.metrics.MetricRegistry@5503b61b
>> [junit4] 2> 2363722 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/
>> 127.0.0.1:45825_solr
>> [junit4] 2> 2363723 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.m.SolrMetricManager Closing metric reporters for
>> registry=solr.cluster, tag=null
>> [junit4] 2> 2363723 INFO (zkCallback-2688-thread-1-
>> processing-n:127.0.0.1:45825_solr) [n:127.0.0.1:45825_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
>> [junit4] 2> 2363723 INFO (zkCallback-2680-thread-1) [ ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
>> [junit4] 2> 2363723 INFO (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
>> [junit4] 2> 2363723 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.c.TransientSolrCoreCacheDefault
>> Allocating transient cache for 2147483647 <(214)%20748-3647> transient
>> cores
>> [junit4] 2> 2363723 INFO (zkCallback-2670-thread-1-
>> processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
>> [junit4] 2> 2363724 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@
>> 46112453{/solr,null,UNAVAILABLE}
>> [junit4] 2> 2363724 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session
>> Stopped scavenging
>> [junit4] 2> 2364709 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
>> NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
>> [junit4] 2> 2364709 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost
>> node: 127.0.0.1:45825_solr
>> [junit4] 2> 2364710 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger
>> firing registered processor for lost nodes: [127.0.0.1:45825_solr]
>> [junit4] 2> 2364710 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++
>> Cooldown inactive - processing event: {
>> [junit4] 2> "id":"5d5a9c17d5001T3me4vw7c8et15q9500dmer1ak",
>> [junit4] 2> "source":"node_lost_trigger1",
>> [junit4] 2> "eventTime":1642299955957761,
>> [junit4] 2> "eventType":"NODELOST",
>> [junit4] 2> "properties":{
>> [junit4] 2> "eventTimes":[1642299955957761],
>> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
>> [junit4] 2> 2364710 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all
>> triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1,
>> node_added_trigger1]
>> [junit4] 2> 2364710 INFO (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection
>> .system does not exist, disabling logging.
>> [junit4] 2> 2364710 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
>> [junit4] 2> "id":"5d5a9c17d5001T3me4vw7c8et15q9500dmer1ak",
>> [junit4] 2> "source":"node_lost_trigger1",
>> [junit4] 2> "eventTime":1642299955957761,
>> [junit4] 2> "eventType":"NODELOST",
>> [junit4] 2> "properties":{
>> [junit4] 2> "eventTimes":[1642299955957761],
>> [junit4] 2> "_enqueue_time_":1642299956256878,
>> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
>> [junit4] 2> 2364711 INFO (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.TriggerIntegrationTest action executed from
>> node_lost_trigger1
>> [junit4] 2> 2364711 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after
>> 5000ms
>> [junit4] 2> 2364711 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after
>> 5000ms
>> [junit4] 2> 2364711 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after
>> 5000ms
>> [junit4] 2> 2364711 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after
>> 5000ms
>> [junit4] 2> 2364711 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers -- processing took 0 ms for event id=
>> 5d5a9c17d5001T3me4vw7c8et15q9500dmer1ak
>> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-2)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
>> NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
>> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
>> NodeAddedTrigger node_added_trigger2
>> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-4)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
>> NodeAddedTrigger node_added_trigger1
>> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-1)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes:
>> 2
>> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-4)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes:
>> 2
>> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
>> NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
>> [junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost
>> node: 127.0.0.1:45825_solr
>> [junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger
>> firing registered processor for lost nodes: [127.0.0.1:45825_solr]
>> [junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++
>> Cooldown inactive - processing event: {
>> [junit4] 2> "id":"5d5aaebb1d434T3me4vw7c8et15q9500dmer1am",
>> [junit4] 2> "source":"node_lost_trigger2",
>> [junit4] 2> "eventTime":1642304959009844,
>> [junit4] 2> "eventType":"NODELOST",
>> [junit4] 2> "properties":{
>> [junit4] 2> "eventTimes":[1642304959009844],
>> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
>> [junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all
>> triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1,
>> node_added_trigger1]
>> [junit4] 2> 2369713 INFO (ScheduledTrigger-4813-thread-3)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection
>> .system does not exist, disabling logging.
>> [junit4] 2> 2369713 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
>> [junit4] 2> "id":"5d5aaebb1d434T3me4vw7c8et15q9500dmer1am",
>> [junit4] 2> "source":"node_lost_trigger2",
>> [junit4] 2> "eventTime":1642304959009844,
>> [junit4] 2> "eventType":"NODELOST",
>> [junit4] 2> "properties":{
>> [junit4] 2> "eventTimes":[1642304959009844],
>> [junit4] 2> "_enqueue_time_":1642304959242817,
>> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
>> [junit4] 2> 2369713 INFO (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.TriggerIntegrationTest last action at 1517187967964000000
>> current time = 1517187972966000000
>> [junit4] 2> real diff: 5002000000 <(500)%20200-0000>
>> [junit4] 2> min diff: 4998000000
>> [junit4] 2> 2369713 INFO (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.TriggerIntegrationTest action executed from
>> node_lost_trigger2
>> [junit4] 2> 2369713 INFO (TEST-TriggerIntegrationTest.
>> testTriggerThrottling-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
>> [junit4] 2> 2369717 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after
>> 5000ms
>> [junit4] 2> 2369717 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after
>> 5000ms
>> [junit4] 2> 2369717 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after
>> 5000ms
>> [junit4] 2> 2369717 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after
>> 5000ms
>> [junit4] 2> 2369718 DEBUG (AutoscalingActionExecutor-
>> 4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr
>> ] o.a.s.c.a.ScheduledTriggers -- processing took 4 ms for event id=
>> 5d5aaebb1d434T3me4vw7c8et15q9500dmer1am
>> [junit4] 2> 2369725 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.SolrTestCaseJ4
>> ###Starting testSearchRate
>> [junit4] 2> 2369725 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.c.a.TriggerIntegrationTest
>> /autoscaling.json reset, new znode version 7
>> [junit4] 2> 2369725 DEBUG (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode
>> version 7
>> [junit4] 2> 2369725 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
>> trigger updates upto znodeVersion 7
>> [junit4] 2> 2369728 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean
>> old nodeLost markers
>> [junit4] 2> 2369728 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean
>> old nodeAdded markers
>> [junit4] 2> 2369728 DEBUG (OverseerAutoScalingTriggerThre
>> ad-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
>> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
>> znodeVersion 7, lastZnodeVersion 7
>> [junit4] 2> 2369729 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server
>> jetty-9.4.8.v20171121, build timestamp: 2017-11-22T09:27:37+12:00, git
>> hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
>> [junit4] 2> 2369729 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session
>> DefaultSessionIdManager workerName=node0
>> [junit4] 2> 2369729 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session No
>> SessionScavenger set, using defaults
>> [junit4] 2> 2369729 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session
>> Scavenging every 660000ms
>> [junit4] 2> 2369729 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.h.ContextHandler
>> Started o.e.j.s.ServletContextHandler@62c45afd{/solr,null,AVAILABLE}
>> [junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.AbstractConnector
>> Started ServerConnector@1472c7a9{SSL,[ssl, http/1.1]}{127.0.0.1:46816}
>> [junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server Started
>> @2380977ms
>> [junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
>> hostPort=46816}
>> [junit4] 2> 2369730 ERROR (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may
>> be missing or incomplete.
>> [junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr?
>> version 7.3.0
>> [junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port
>> null
>> [junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
>> [junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
>> 2018-01-29T01:06:12.983Z
>> [junit4] 2> 2369731 INFO (zkConnectionManagerCallback-2693-thread-1)
>> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2369732 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ]
>> o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
>> [junit4] 2> 2369738 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.c.ZkContainer
>> Zookeeper client=127.0.0.1:46316/solr
>> [junit4] 2> 2369739 INFO (zkConnectionManagerCallback-2697-thread-1)
>> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2369740 INFO (zkConnectionManagerCallback-
>> 2699-thread-1-processing-n:127.0.0.1:46816_solr) [n:127.0.0.1:46816_solr
>> ] o.a.s.c.c.ConnectionManager zkClient has connected
>> [junit4] 2> 2369744 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:46816_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
>> [junit4] 2> 2369745 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:46816_solr ]
>> o.a.s.c.Overseer Overseer (id=null) closing
>> [junit4] 2> 2369746 INFO (TEST-TriggerIntegrationTest.
>> testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:46816_solr ]
>> o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/
>> 127.0.0.1:46816_solr
>> [junit4] 2> 2369746 INFO (zkCallback-2680-thread-1) [ ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
>> [junit4] 2> 2369746 INFO (zkCallback-2668-thread-1-
>> processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
>> [junit4] 2> 2369746 INFO (zkCallback-2670-thread-1-
>> processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
>> [junit4] 2> 2369754 INFO (zkCallback-2698-thread-1-
>> processing-n:127.0.0.1:46816_solr) [n:127.0.0.1:46816_solr ]
>> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
>> [junit4]
>
>
>
> --
> Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
> LinkedIn: http://linkedin.com/in/davidwsmiley | Book: http://www.
> solrenterprisesearchserver.com
>
--
http://www.the111shift.com
Re: [JENKINS] Lucene-Solr-Tests-7.x - Build # 339 - Still Unstable
Posted by David Smiley <da...@gmail.com>.
RE failure of AliasIntegrationTest I think I see the problem. Gus, note
that you called processAsync(...) instead of process(...) in tearDown when
deleting the aliases. Perhaps it's too easy to call the async version :-)
I'll fix.
On Sun, Jan 28, 2018 at 9:17 PM Apache Jenkins Server <
jenkins@builds.apache.org> wrote:
> Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/339/
>
> 3 tests failed.
> FAILED: org.apache.solr.cloud.AliasIntegrationTest.testMetadata
>
> Error Message:
> Error from server at http://127.0.0.1:37832/solr: Collection :
> collection2meta is part of alias meta3 remove or modify the alias before
> removing this collection.
>
> Stack Trace:
> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:
> Error from server at http://127.0.0.1:37832/solr: Collection :
> collection2meta is part of alias meta3 remove or modify the alias before
> removing this collection.
> at
> __randomizedtesting.SeedInfo.seed([DA442618305C7D2C:D3546C1C77B52145]:0)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
> at
> org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
> at
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
> at
> org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
> at
> org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
> at
> org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
> at
> org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
> at
> org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
> at
> org.apache.solr.cloud.AliasIntegrationTest.tearDown(AliasIntegrationTest.java:95)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.lang.Thread.run(Thread.java:748)
>
>
> FAILED:
> org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventFromRestoredState
>
> Error Message:
>
>
> Stack Trace:
> java.lang.AssertionError
> at
> __randomizedtesting.SeedInfo.seed([DA442618305C7D2C:DA7292D81951D946]:0)
> at org.junit.Assert.fail(Assert.java:92)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertTrue(Assert.java:54)
> at
> org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventFromRestoredState(TriggerIntegrationTest.java:760)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.lang.Thread.run(Thread.java:748)
>
>
> FAILED:
> org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
>
> Error Message:
>
>
> Stack Trace:
> java.lang.AssertionError
> at
> __randomizedtesting.SeedInfo.seed([DA442618305C7D2C:604811976FB4AB63]:0)
> at org.junit.Assert.fail(Assert.java:92)
> at org.junit.Assert.assertTrue(Assert.java:43)
> at org.junit.Assert.assertNull(Assert.java:551)
> at org.junit.Assert.assertNull(Assert.java:562)
> at
> org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
> at
> com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
> at
> org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
> at
> org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
> at
> org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
> at
> org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
> at
> com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
> at
> com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
> at java.lang.Thread.run(Thread.java:748)
>
>
>
>
> Build Log:
> [...truncated 12951 lines...]
> [junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
> [junit4] 2> 2357318 INFO
> (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ]
> o.a.s.SolrTestCaseJ4 SecureRandom sanity checks:
> test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
> [junit4] 2> Creating dataDir:
> /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/init-core-data-001
> [junit4] 2> 2357319 INFO
> (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true)
> w/NUMERIC_DOCVALUES_SYSPROP=true
> [junit4] 2> 2357319 INFO
> (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ]
> o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via:
> @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN,
> clientAuth=NaN)
> [junit4] 2> 2357320 INFO
> (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ]
> o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in
> /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001
> [junit4] 2> 2357320 INFO
> (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ]
> o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
> [junit4] 2> 2357320 INFO (Thread-4693) [ ] o.a.s.c.ZkTestServer
> client port:0.0.0.0/0.0.0.0:0
> [junit4] 2> 2357320 INFO (Thread-4693) [ ] o.a.s.c.ZkTestServer
> Starting server
> [junit4] 2> 2357322 ERROR (Thread-4693) [ ]
> o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper
> server won't take any action on ERROR or SHUTDOWN server state changes
> [junit4] 2> 2357421 INFO
> (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ]
> o.a.s.c.ZkTestServer start zk server on port:46316
> [junit4] 2> 2357422 INFO (zkConnectionManagerCallback-2655-thread-1)
> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
> [junit4] 2> 2357425 INFO (jetty-launcher-2652-thread-2) [ ]
> o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp:
> 2017-11-22T09:27:37+12:00, git hash:
> 82b8fb23f757335bb3329d540ce37a2a2615f0a8
> [junit4] 2> 2357425 INFO (jetty-launcher-2652-thread-1) [ ]
> o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp:
> 2017-11-22T09:27:37+12:00, git hash:
> 82b8fb23f757335bb3329d540ce37a2a2615f0a8
> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-2) [ ]
> o.e.j.s.session DefaultSessionIdManager workerName=node0
> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-1) [ ]
> o.e.j.s.session DefaultSessionIdManager workerName=node0
> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-2) [ ]
> o.e.j.s.session No SessionScavenger set, using defaults
> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-1) [ ]
> o.e.j.s.session No SessionScavenger set, using defaults
> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-2) [ ]
> o.e.j.s.session Scavenging every 600000ms
> [junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-1) [ ]
> o.e.j.s.session Scavenging every 600000ms
> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ]
> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@676003ea
> {/solr,null,AVAILABLE}
> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ]
> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@275f23bf
> {/solr,null,AVAILABLE}
> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ]
> o.e.j.s.AbstractConnector Started ServerConnector@655c52be{SSL,[ssl,
> http/1.1]}{127.0.0.1:35199}
> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ]
> o.e.j.s.AbstractConnector Started ServerConnector@190ae2b1{SSL,[ssl,
> http/1.1]}{127.0.0.1:35307}
> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ]
> o.e.j.s.Server Started @2368674ms
> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ]
> o.e.j.s.Server Started @2368674ms
> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ]
> o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
> hostPort=35199}
> [junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ]
> o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
> hostPort=35307}
> [junit4] 2> 2357427 ERROR (jetty-launcher-2652-thread-1) [ ]
> o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may
> be missing or incomplete.
> [junit4] 2> 2357427 ERROR (jetty-launcher-2652-thread-2) [ ]
> o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may
> be missing or incomplete.
> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ]
> o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr?
> version 7.3.0
> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ]
> o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr?
> version 7.3.0
> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ]
> o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port
> null
> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ]
> o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port
> null
> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ]
> o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ]
> o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ]
> o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
> 2018-01-29T01:06:00.681Z
> [junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ]
> o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
> 2018-01-29T01:06:00.681Z
> [junit4] 2> 2357435 INFO (zkConnectionManagerCallback-2658-thread-1)
> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
> [junit4] 2> 2357436 INFO (jetty-launcher-2652-thread-1) [ ]
> o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
> [junit4] 2> 2357440 INFO (zkConnectionManagerCallback-2659-thread-1)
> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
> [junit4] 2> 2357441 INFO (jetty-launcher-2652-thread-2) [ ]
> o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
> [junit4] 2> 2357445 INFO (jetty-launcher-2652-thread-1) [ ]
> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
> [junit4] 2> 2357445 INFO (jetty-launcher-2652-thread-2) [ ]
> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
> [junit4] 2> 2357446 INFO (zkConnectionManagerCallback-2663-thread-1)
> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
> [junit4] 2> 2357452 INFO (zkConnectionManagerCallback-2667-thread-1)
> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
> [junit4] 2> 2357453 INFO
> (zkConnectionManagerCallback-2669-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.c.ConnectionManager zkClient has
> connected
> [junit4] 2> 2357453 INFO
> (zkConnectionManagerCallback-2671-thread-1-processing-n:127.0.0.1:35307_solr)
> [n:127.0.0.1:35307_solr ] o.a.s.c.c.ConnectionManager zkClient has
> connected
> [junit4] 2> 2357519 INFO (jetty-launcher-2652-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 2357519 INFO (jetty-launcher-2652-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.c.OverseerElectionContext I am going
> to be the leader 127.0.0.1:35199_solr
> [junit4] 2> 2357520 INFO (jetty-launcher-2652-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.c.Overseer Overseer
> (id=99430430184177669-127.0.0.1:35199_solr-n_0000000000) starting
> [junit4] 2> 2357521 INFO (jetty-launcher-2652-thread-2)
> [n:127.0.0.1:35307_solr ] o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 2357521 INFO (jetty-launcher-2652-thread-2)
> [n:127.0.0.1:35307_solr ] o.a.s.c.ZkController Register node as live
> in ZooKeeper:/live_nodes/127.0.0.1:35307_solr
> [junit4] 2> 2357534 INFO
> (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr)
> [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (0) -> (1)
> [junit4] 2> 2357535 INFO
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (0) -> (1)
> [junit4] 2> 2357537 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Adding
> .autoAddReplicas trigger
> [junit4] 2> 2357538 INFO (jetty-launcher-2652-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.c.ZkController Register node as live
> in ZooKeeper:/live_nodes/127.0.0.1:35199_solr
> [junit4] 2> 2357538 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing
> /autoscaling.json with znode version 1
> [junit4] 2> 2357546 INFO
> (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr)
> [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (1) -> (2)
> [junit4] 2> 2357546 INFO
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (1) -> (2)
> [junit4] 2> 2357546 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes:
> [127.0.0.1:35307_solr, 127.0.0.1:35199_solr]
> [junit4] 2> 2357546 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
> znodeVersion 1, lastZnodeVersion -1
> [junit4] 2> 2357546 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
> trigger updates upto znodeVersion 1
> [junit4] 2> 2357547 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old
> nodeAdded markers
> [junit4] 2> 2357547 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
> znodeVersion 1, lastZnodeVersion 1
> [junit4] 2> 2357548 DEBUG (ScheduledTrigger-4813-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
> NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
> [junit4] 2> 2357581 INFO (jetty-launcher-2652-thread-2)
> [n:127.0.0.1:35307_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr_35307.solr.node' (registry 'solr.node') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
> [junit4] 2> 2357582 INFO (jetty-launcher-2652-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr_35199.solr.node' (registry 'solr.node') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
> [junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-2)
> [n:127.0.0.1:35307_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr_35307.solr.jvm' (registry 'solr.jvm') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
> [junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-2)
> [n:127.0.0.1:35307_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr_35307.solr.jetty' (registry 'solr.jetty') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
> [junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr_35199.solr.jvm' (registry 'solr.jvm') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
> [junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr_35199.solr.jetty' (registry 'solr.jetty') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
> [junit4] 2> 2357590 INFO (jetty-launcher-2652-thread-2)
> [n:127.0.0.1:35307_solr ] o.a.s.c.CorePropertiesLocator Found 0 core
> definitions underneath
> /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001/node2/.
> [junit4] 2> 2357590 INFO (jetty-launcher-2652-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.c.CorePropertiesLocator Found 0 core
> definitions underneath
> /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001/node1/.
> [junit4] 2> 2357622 INFO (zkConnectionManagerCallback-2677-thread-1)
> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
> [junit4] 2> 2357624 INFO (zkConnectionManagerCallback-2681-thread-1)
> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
> [junit4] 2> 2357624 INFO
> (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ]
> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
> [junit4] 2> 2357625 INFO
> (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ]
> o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46316/solr
> ready
> [junit4] 2> 2357637 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
> [junit4] 2> 2357637 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode
> version 2
> [junit4] 2> 2357637 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing
> /autoscaling.json with znode version 2
> [junit4] 2> 2357637 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
> trigger updates upto znodeVersion 2
> [junit4] 2> 2357638 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old
> nodeLost markers
> [junit4] 2> 2357638 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old
> nodeAdded markers
> [junit4] 2> 2357639 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
> znodeVersion 2, lastZnodeVersion 2
> [junit4] 2> 2357664 INFO (qtp1081540318-14383) [n:127.0.0.1:35199_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
> status=0 QTime=0
> [junit4] 2> 2357665 INFO (qtp702010038-14390) [n:127.0.0.1:35307_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
> status=0 QTime=0
> [junit4] 2> 2357666 DEBUG (qtp702010038-14382) [n:127.0.0.1:35307_solr
> ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
> [junit4] 2> 2357666 INFO (qtp702010038-14382) [n:127.0.0.1:35307_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling
> params={wt=javabin&version=2} status=0 QTime=14
> [junit4] 2> 2357666 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing
> /autoscaling.json with znode version 3
> [junit4] 2> 2357669 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial
> livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr]
> [junit4] 2> 2357669 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger
> node_added_trigger1 instantiated with properties: {event=nodeAdded,
> waitFor=0, actions=[{name=test,
> class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}],
> enabled=true}
> [junit4] 2> 2357669 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
> trigger updates upto znodeVersion 3
> [junit4] 2> 2357669 INFO (qtp1081540318-14387) [n:127.0.0.1:35199_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
> status=0 QTime=0
> [junit4] 2> 2357670 INFO (qtp702010038-14384) [n:127.0.0.1:35307_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
> status=0 QTime=0
> [junit4] 2> 2357670 INFO
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest
> TestTriggerAction init
> [junit4] 2> 2357670 DEBUG (qtp702010038-14386) [n:127.0.0.1:35307_solr
> ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
> [junit4] 2> 2357671 INFO (qtp702010038-14386) [n:127.0.0.1:35307_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling
> params={wt=javabin&version=2} status=0 QTime=2
> [junit4] 2> 2357678 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing
> /autoscaling.json with znode version 4
> [junit4] 2> 2357678 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial
> livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr]
> [junit4] 2> 2357678 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger
> node_added_trigger2 instantiated with properties: {event=nodeAdded,
> waitFor=0, actions=[{name=test,
> class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}],
> enabled=true}
> [junit4] 2> 2357678 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial
> livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr]
> [junit4] 2> 2357679 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger
> node_added_trigger1 instantiated with properties: {event=nodeAdded,
> waitFor=0, actions=[{name=test,
> class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}],
> enabled=true}
> [junit4] 2> 2357679 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old
> nodeLost markers
> [junit4] 2> 2357679 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
> znodeVersion 4, lastZnodeVersion 3
> [junit4] 2> 2357679 DEBUG (ScheduledTrigger-4813-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
> NodeAddedTrigger node_added_trigger1
> [junit4] 2> 2357679 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
> trigger updates upto znodeVersion 4
> [junit4] 2> 2357679 DEBUG (ScheduledTrigger-4813-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
> [junit4] 2> 2357680 INFO
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest
> TestTriggerAction init
> [junit4] 2> 2357681 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old
> nodeLost markers
> [junit4] 2> 2357681 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
> znodeVersion 4, lastZnodeVersion 4
> [junit4] 2> 2357681 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
> NodeAddedTrigger node_added_trigger2
> [junit4] 2> 2357681 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
> [junit4] 2> 2357681 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp:
> 2017-11-22T09:27:37+12:00, git hash:
> 82b8fb23f757335bb3329d540ce37a2a2615f0a8
> [junit4] 2> 2357682 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
> [junit4] 2> 2357682 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.e.j.s.session No SessionScavenger set, using defaults
> [junit4] 2> 2357682 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.e.j.s.session Scavenging every 600000ms
> [junit4] 2> 2357682 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.e.j.s.h.ContextHandler Started
> o.e.j.s.ServletContextHandler@46112453{/solr,null,AVAILABLE}
> [junit4] 2> 2357683 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.e.j.s.AbstractConnector Started ServerConnector@435c6f3d{SSL,[ssl,
> http/1.1]}{127.0.0.1:45825}
> [junit4] 2> 2357683 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.e.j.s.Server Started @2368930ms
> [junit4] 2> 2357683 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
> hostPort=45825}
> [junit4] 2> 2357683 ERROR
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir.
> Logging may be missing or incomplete.
> [junit4] 2> 2357683 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr?
> version 7.3.0
> [junit4] 2> 2357683 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode
> on port null
> [junit4] 2> 2357683 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
> [junit4] 2> 2357683 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
> 2018-01-29T01:06:00.936Z
> [junit4] 2> 2357684 INFO (zkConnectionManagerCallback-2683-thread-1)
> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
> [junit4] 2> 2357685 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
> [junit4] 2> 2357691 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
> [junit4] 2> 2357692 INFO (zkConnectionManagerCallback-2687-thread-1)
> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
> [junit4] 2> 2357693 INFO
> (zkConnectionManagerCallback-2689-thread-1-processing-n:127.0.0.1:45825_solr)
> [n:127.0.0.1:45825_solr ] o.a.s.c.c.ConnectionManager zkClient has
> connected
> [junit4] 2> 2357697 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [n:127.0.0.1:45825_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (0) -> (2)
> [junit4] 2> 2357698 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [n:127.0.0.1:45825_solr ] o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 2357699 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [n:127.0.0.1:45825_solr ] o.a.s.c.ZkController Register node as live
> in ZooKeeper:/live_nodes/127.0.0.1:45825_solr
> [junit4] 2> 2357700 INFO
> (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr)
> [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (2) -> (3)
> [junit4] 2> 2357700 INFO
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (2) -> (3)
> [junit4] 2> 2357700 INFO
> (zkCallback-2688-thread-1-processing-n:127.0.0.1:45825_solr)
> [n:127.0.0.1:45825_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (2) -> (3)
> [junit4] 2> 2357700 INFO (zkCallback-2680-thread-1) [ ]
> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
> [junit4] 2> 2357740 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [n:127.0.0.1:45825_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr_45825.solr.node' (registry 'solr.node') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
> [junit4] 2> 2357749 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [n:127.0.0.1:45825_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr_45825.solr.jvm' (registry 'solr.jvm') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
> [junit4] 2> 2357749 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [n:127.0.0.1:45825_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for
> 'solr_45825.solr.jetty' (registry 'solr.jetty') enabled at server:
> com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
> [junit4] 2> 2357750 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [n:127.0.0.1:45825_solr ] o.a.s.c.CorePropertiesLocator Found 0 core
> definitions underneath
> /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001/node3/.
> [junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
> NodeAddedTrigger node_added_trigger1
> [junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
> [junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new
> node: 127.0.0.1:45825_solr at time 1642293927204014
> [junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger
> node_added_trigger1 firing registered processor for nodes: [127.0.0.1:45825_solr]
> added at times [1642293927204014], now=1642293927367265
> [junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++
> Cooldown inactive - processing event: {
> [junit4] 2> "id":"5d5a85a25d4aeT3me4vw7c8et15q9500dmer1ag",
> [junit4] 2> "source":"node_added_trigger1",
> [junit4] 2> "eventTime":1642293927204014,
> [junit4] 2> "eventType":"NODEADDED",
> [junit4] 2> "properties":{
> [junit4] 2> "eventTimes":[1642293927204014],
> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
> [junit4] 2> 2358682 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all
> triggers: [node_added_trigger2, node_added_trigger1]
> [junit4] 2> 2358683 INFO (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection
> .system does not exist, disabling logging.
> [junit4] 2> 2358683 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing
> actions for {
> [junit4] 2> "id":"5d5a85a25d4aeT3me4vw7c8et15q9500dmer1ag",
> [junit4] 2> "source":"node_added_trigger1",
> [junit4] 2> "eventTime":1642293927204014,
> [junit4] 2> "eventType":"NODEADDED",
> [junit4] 2> "properties":{
> [junit4] 2> "eventTimes":[1642293927204014],
> [junit4] 2> "_enqueue_time_":1642293928447431,
> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
> [junit4] 2> 2358683 INFO
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest action
> executed from node_added_trigger1
> [junit4] 2> 2358684 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_added_trigger2 after 5000ms
> [junit4] 2> 2358684 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_added_trigger1 after 5000ms
> [junit4] 2> 2358684 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing
> took 1 ms for event id=5d5a85a25d4aeT3me4vw7c8et15q9500dmer1ag
> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
> NodeAddedTrigger node_added_trigger2
> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-4)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
> NodeAddedTrigger node_added_trigger1
> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-4)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new
> node: 127.0.0.1:45825_solr at time 1642298930755018
> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger
> node_added_trigger2 firing registered processor for nodes: [127.0.0.1:45825_solr]
> added at times [1642298930755018], now=1642298930823201
> [junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++
> Cooldown inactive - processing event: {
> [junit4] 2> "id":"5d5a98461f5caT3me4vw7c8et15q9500dmer1ai",
> [junit4] 2> "source":"node_added_trigger2",
> [junit4] 2> "eventTime":1642298930755018,
> [junit4] 2> "eventType":"NODEADDED",
> [junit4] 2> "properties":{
> [junit4] 2> "eventTimes":[1642298930755018],
> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
> [junit4] 2> 2363685 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all
> triggers: [node_added_trigger2, node_added_trigger1]
> [junit4] 2> 2363685 INFO (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection
> .system does not exist, disabling logging.
> [junit4] 2> 2363685 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing
> actions for {
> [junit4] 2> "id":"5d5a98461f5caT3me4vw7c8et15q9500dmer1ai",
> [junit4] 2> "source":"node_added_trigger2",
> [junit4] 2> "eventTime":1642298930755018,
> [junit4] 2> "eventType":"NODEADDED",
> [junit4] 2> "properties":{
> [junit4] 2> "eventTimes":[1642298930755018],
> [junit4] 2> "_enqueue_time_":1642298931021681,
> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
> [junit4] 2> 2363685 INFO
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest last action
> at 1517187961936000000 current time = 1517187966938000000
> [junit4] 2> real diff: 5002000000 <(500)%20200-0000>
> [junit4] 2> min diff: 4998000000
> [junit4] 2> 2363685 INFO
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest action
> executed from node_added_trigger2
> [junit4] 2> 2363686 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_added_trigger2 after 5000ms
> [junit4] 2> 2363686 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_added_trigger1 after 5000ms
> [junit4] 2> 2363686 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing
> took 0 ms for event id=5d5a98461f5caT3me4vw7c8et15q9500dmer1ai
> [junit4] 2> 2363689 INFO (qtp1081540318-14383) [n:127.0.0.1:35199_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
> status=0 QTime=0
> [junit4] 2> 2363691 INFO (qtp702010038-14390) [n:127.0.0.1:35307_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
> status=0 QTime=0
> [junit4] 2> 2363705 INFO (qtp2109945748-14443) [n:127.0.0.1:45825_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
> status=0 QTime=0
> [junit4] 2> 2363705 DEBUG (qtp702010038-14382) [n:127.0.0.1:35307_solr
> ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
> [junit4] 2> 2363706 INFO (qtp702010038-14382) [n:127.0.0.1:35307_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling
> params={wt=javabin&version=2} status=0 QTime=18
> [junit4] 2> 2363706 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing
> /autoscaling.json with znode version 5
> [junit4] 2> 2363706 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial
> livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825
> _solr]
> [junit4] 2> 2363706 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger
> node_added_trigger2 instantiated with properties: {event=nodeAdded,
> waitFor=0, actions=[{name=test,
> class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}],
> enabled=true}
> [junit4] 2> 2363706 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes:
> [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
> [junit4] 2> 2363707 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial
> livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825
> _solr]
> [junit4] 2> 2363707 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger
> node_added_trigger1 instantiated with properties: {event=nodeAdded,
> waitFor=0, actions=[{name=test,
> class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}],
> enabled=true}
> [junit4] 2> 2363707 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
> trigger updates upto znodeVersion 5
> [junit4] 2> 2363707 INFO (qtp1081540318-14387) [n:127.0.0.1:35199_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
> status=0 QTime=0
> [junit4] 2> 2363707 INFO (qtp702010038-14384) [n:127.0.0.1:35307_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
> status=0 QTime=0
> [junit4] 2> 2363708 INFO
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest
> TestTriggerAction init
> [junit4] 2> 2363708 INFO (qtp2109945748-14445) [n:127.0.0.1:45825_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics
> params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core}
> status=0 QTime=0
> [junit4] 2> 2363708 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
> znodeVersion 5, lastZnodeVersion 5
> [junit4] 2> 2363708 DEBUG (qtp702010038-14386) [n:127.0.0.1:35307_solr
> ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
> [junit4] 2> 2363708 DEBUG (ScheduledTrigger-4813-thread-4)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
> NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
> [junit4] 2> 2363708 INFO (qtp702010038-14386) [n:127.0.0.1:35307_solr
> ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling
> params={wt=javabin&version=2} status=0 QTime=2
> [junit4] 2> 2363709 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing
> /autoscaling.json with znode version 6
> [junit4] 2> 2363709 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial
> livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825
> _solr]
> [junit4] 2> 2363709 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger
> node_added_trigger2 instantiated with properties: {event=nodeAdded,
> waitFor=0, actions=[{name=test,
> class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}],
> enabled=true}
> [junit4] 2> 2363709 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes:
> [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
> [junit4] 2> 2363709 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial
> livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825
> _solr]
> [junit4] 2> 2363709 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger
> node_added_trigger1 instantiated with properties: {event=nodeAdded,
> waitFor=0, actions=[{name=test,
> class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}],
> enabled=true}
> [junit4] 2> 2363710 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes:
> [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
> [junit4] 2> 2363710 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
> trigger updates upto znodeVersion 6
> [junit4] 2> 2363711 INFO
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest
> TestTriggerAction init
> [junit4] 2> 2363711 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
> znodeVersion 6, lastZnodeVersion 6
> [junit4] 2> 2363712 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@435c6f3d{SSL,[ssl,
> http/1.1]}{127.0.0.1:0}
> [junit4] 2> 2363712 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1700547441
> [junit4] 2> 2363712 DEBUG (ScheduledTrigger-4813-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
> NodeLostTrigger: node_lost_trigger2 with currently live nodes: 3
> [junit4] 2> 2363712 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.m.SolrMetricManager Closing metric reporters for
> registry=solr.node, tag=null
> [junit4] 2> 2363712 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.m.r.SolrJmxReporter Closing reporter
> [org.apache.solr.metrics.reporters.SolrJmxReporter@4eb5a754: rootName =
> solr_45825, domain = solr.node, service url = null, agent id = null] for
> registry solr.node / com.codahale.metrics.MetricRegistry@3907f8dd
> [junit4] 2> 2363717 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.m.SolrMetricManager Closing metric reporters for
> registry=solr.jvm, tag=null
> [junit4] 2> 2363717 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.m.r.SolrJmxReporter Closing reporter
> [org.apache.solr.metrics.reporters.SolrJmxReporter@103306db: rootName =
> solr_45825, domain = solr.jvm, service url = null, agent id = null] for
> registry solr.jvm / com.codahale.metrics.MetricRegistry@dc0593e
> [junit4] 2> 2363721 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.m.SolrMetricManager Closing metric reporters for
> registry=solr.jetty, tag=null
> [junit4] 2> 2363721 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.m.r.SolrJmxReporter Closing reporter
> [org.apache.solr.metrics.reporters.SolrJmxReporter@23607bff: rootName =
> solr_45825, domain = solr.jetty, service url = null, agent id = null] for
> registry solr.jetty / com.codahale.metrics.MetricRegistry@5503b61b
> [junit4] 2> 2363722 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/
> 127.0.0.1:45825_solr
> [junit4] 2> 2363723 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.m.SolrMetricManager Closing metric reporters for
> registry=solr.cluster, tag=null
> [junit4] 2> 2363723 INFO
> (zkCallback-2688-thread-1-processing-n:127.0.0.1:45825_solr)
> [n:127.0.0.1:45825_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (3) -> (2)
> [junit4] 2> 2363723 INFO (zkCallback-2680-thread-1) [ ]
> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
> [junit4] 2> 2363723 INFO
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (3) -> (2)
> [junit4] 2> 2363723 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for
> 2147483647 <(214)%20748-3647> transient cores
> [junit4] 2> 2363723 INFO
> (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr)
> [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (3) -> (2)
> [junit4] 2> 2363724 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.e.j.s.h.ContextHandler Stopped
> o.e.j.s.ServletContextHandler@46112453{/solr,null,UNAVAILABLE}
> [junit4] 2> 2363724 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.e.j.s.session Stopped scavenging
> [junit4] 2> 2364709 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
> NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
> [junit4] 2> 2364709 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost
> node: 127.0.0.1:45825_solr
> [junit4] 2> 2364710 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger
> firing registered processor for lost nodes: [127.0.0.1:45825_solr]
> [junit4] 2> 2364710 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++
> Cooldown inactive - processing event: {
> [junit4] 2> "id":"5d5a9c17d5001T3me4vw7c8et15q9500dmer1ak",
> [junit4] 2> "source":"node_lost_trigger1",
> [junit4] 2> "eventTime":1642299955957761,
> [junit4] 2> "eventType":"NODELOST",
> [junit4] 2> "properties":{
> [junit4] 2> "eventTimes":[1642299955957761],
> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
> [junit4] 2> 2364710 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all
> triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1,
> node_added_trigger1]
> [junit4] 2> 2364710 INFO (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection
> .system does not exist, disabling logging.
> [junit4] 2> 2364710 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing
> actions for {
> [junit4] 2> "id":"5d5a9c17d5001T3me4vw7c8et15q9500dmer1ak",
> [junit4] 2> "source":"node_lost_trigger1",
> [junit4] 2> "eventTime":1642299955957761,
> [junit4] 2> "eventType":"NODELOST",
> [junit4] 2> "properties":{
> [junit4] 2> "eventTimes":[1642299955957761],
> [junit4] 2> "_enqueue_time_":1642299956256878,
> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
> [junit4] 2> 2364711 INFO
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest action
> executed from node_lost_trigger1
> [junit4] 2> 2364711 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_lost_trigger2 after 5000ms
> [junit4] 2> 2364711 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_added_trigger2 after 5000ms
> [junit4] 2> 2364711 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_lost_trigger1 after 5000ms
> [junit4] 2> 2364711 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_added_trigger1 after 5000ms
> [junit4] 2> 2364711 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing
> took 0 ms for event id=5d5a9c17d5001T3me4vw7c8et15q9500dmer1ak
> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-2)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
> NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
> NodeAddedTrigger node_added_trigger2
> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-4)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running
> NodeAddedTrigger node_added_trigger1
> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-1)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-4)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
> [junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running
> NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
> [junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost
> node: 127.0.0.1:45825_solr
> [junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger
> firing registered processor for lost nodes: [127.0.0.1:45825_solr]
> [junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++
> Cooldown inactive - processing event: {
> [junit4] 2> "id":"5d5aaebb1d434T3me4vw7c8et15q9500dmer1am",
> [junit4] 2> "source":"node_lost_trigger2",
> [junit4] 2> "eventTime":1642304959009844,
> [junit4] 2> "eventType":"NODELOST",
> [junit4] 2> "properties":{
> [junit4] 2> "eventTimes":[1642304959009844],
> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
> [junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all
> triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1,
> node_added_trigger1]
> [junit4] 2> 2369713 INFO (ScheduledTrigger-4813-thread-3)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection
> .system does not exist, disabling logging.
> [junit4] 2> 2369713 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing
> actions for {
> [junit4] 2> "id":"5d5aaebb1d434T3me4vw7c8et15q9500dmer1am",
> [junit4] 2> "source":"node_lost_trigger2",
> [junit4] 2> "eventTime":1642304959009844,
> [junit4] 2> "eventType":"NODELOST",
> [junit4] 2> "properties":{
> [junit4] 2> "eventTimes":[1642304959009844],
> [junit4] 2> "_enqueue_time_":1642304959242817,
> [junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
> [junit4] 2> 2369713 INFO
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest last action
> at 1517187967964000000 current time = 1517187972966000000
> [junit4] 2> real diff: 5002000000 <(500)%20200-0000>
> [junit4] 2> min diff: 4998000000
> [junit4] 2> 2369713 INFO
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest action
> executed from node_lost_trigger2
> [junit4] 2> 2369713 INFO
> (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C])
> [ ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
> [junit4] 2> 2369717 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_lost_trigger2 after 5000ms
> [junit4] 2> 2369717 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_added_trigger2 after 5000ms
> [junit4] 2> 2369717 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_lost_trigger1 after 5000ms
> [junit4] 2> 2369717 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming
> trigger: node_added_trigger1 after 5000ms
> [junit4] 2> 2369718 DEBUG
> (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing
> took 4 ms for event id=5d5aaebb1d434T3me4vw7c8et15q9500dmer1am
> [junit4] 2> 2369725 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
> [junit4] 2> 2369725 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version
> 7
> [junit4] 2> 2369725 DEBUG
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing
> /autoscaling.json with znode version 7
> [junit4] 2> 2369725 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed
> trigger updates upto znodeVersion 7
> [junit4] 2> 2369728 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old
> nodeLost markers
> [junit4] 2> 2369728 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old
> nodeAdded markers
> [junit4] 2> 2369728 DEBUG
> (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000)
> [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current
> znodeVersion 7, lastZnodeVersion 7
> [junit4] 2> 2369729 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp:
> 2017-11-22T09:27:37+12:00, git hash:
> 82b8fb23f757335bb3329d540ce37a2a2615f0a8
> [junit4] 2> 2369729 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.e.j.s.session DefaultSessionIdManager workerName=node0
> [junit4] 2> 2369729 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.e.j.s.session No SessionScavenger set, using defaults
> [junit4] 2> 2369729 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.e.j.s.session Scavenging every 660000ms
> [junit4] 2> 2369729 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62c45afd
> {/solr,null,AVAILABLE}
> [junit4] 2> 2369730 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.e.j.s.AbstractConnector Started ServerConnector@1472c7a9{SSL,[ssl,
> http/1.1]}{127.0.0.1:46816}
> [junit4] 2> 2369730 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.e.j.s.Server Started @2380977ms
> [junit4] 2> 2369730 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr,
> hostPort=46816}
> [junit4] 2> 2369730 ERROR
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may
> be missing or incomplete.
> [junit4] 2> 2369730 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr?
> version 7.3.0
> [junit4] 2> 2369730 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port
> null
> [junit4] 2> 2369730 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
> [junit4] 2> 2369730 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time:
> 2018-01-29T01:06:12.983Z
> [junit4] 2> 2369731 INFO (zkConnectionManagerCallback-2693-thread-1)
> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
> [junit4] 2> 2369732 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
> [junit4] 2> 2369738 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ]
> o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
> [junit4] 2> 2369739 INFO (zkConnectionManagerCallback-2697-thread-1)
> [ ] o.a.s.c.c.ConnectionManager zkClient has connected
> [junit4] 2> 2369740 INFO
> (zkConnectionManagerCallback-2699-thread-1-processing-n:127.0.0.1:46816_solr)
> [n:127.0.0.1:46816_solr ] o.a.s.c.c.ConnectionManager zkClient has
> connected
> [junit4] 2> 2369744 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C])
> [n:127.0.0.1:46816_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (0) -> (2)
> [junit4] 2> 2369745 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C])
> [n:127.0.0.1:46816_solr ] o.a.s.c.Overseer Overseer (id=null) closing
> [junit4] 2> 2369746 INFO
> (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C])
> [n:127.0.0.1:46816_solr ] o.a.s.c.ZkController Register node as live
> in ZooKeeper:/live_nodes/127.0.0.1:46816_solr
> [junit4] 2> 2369746 INFO (zkCallback-2680-thread-1) [ ]
> o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
> [junit4] 2> 2369746 INFO
> (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr)
> [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (2) -> (3)
> [junit4] 2> 2369746 INFO
> (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr)
> [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (2) -> (3)
> [junit4] 2> 2369754 INFO
> (zkCallback-2698-thread-1-processing-n:127.0.0.1:46816_solr)
> [n:127.0.0.1:46816_solr ] o.a.s.c.c.ZkStateReader Updated live nodes
> from ZooKeeper... (2) -> (3)
> [junit4]
--
Lucene/Solr Search Committer, Consultant, Developer, Author, Speaker
LinkedIn: http://linkedin.com/in/davidwsmiley | Book:
http://www.solrenterprisesearchserver.com
[JENKINS] Lucene-Solr-Tests-7.x - Build # 339 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/339/
3 tests failed.
FAILED: org.apache.solr.cloud.AliasIntegrationTest.testMetadata
Error Message:
Error from server at http://127.0.0.1:37832/solr: Collection : collection2meta is part of alias meta3 remove or modify the alias before removing this collection.
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:37832/solr: Collection : collection2meta is part of alias meta3 remove or modify the alias before removing this collection.
at __randomizedtesting.SeedInfo.seed([DA442618305C7D2C:D3546C1C77B52145]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AliasIntegrationTest.tearDown(AliasIntegrationTest.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:992)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventFromRestoredState
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([DA442618305C7D2C:DA7292D81951D946]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventFromRestoredState(TriggerIntegrationTest.java:760)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([DA442618305C7D2C:604811976FB4AB63]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.junit.Assert.assertNull(Assert.java:562)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12951 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> 2357318 INFO (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/init-core-data-001
[junit4] 2> 2357319 INFO (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2357319 INFO (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2357320 INFO (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001
[junit4] 2> 2357320 INFO (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2357320 INFO (Thread-4693) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2357320 INFO (Thread-4693) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2357322 ERROR (Thread-4693) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2357421 INFO (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46316
[junit4] 2> 2357422 INFO (zkConnectionManagerCallback-2655-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357425 INFO (jetty-launcher-2652-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2357425 INFO (jetty-launcher-2652-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2357426 INFO (jetty-launcher-2652-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@676003ea{/solr,null,AVAILABLE}
[junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@275f23bf{/solr,null,AVAILABLE}
[junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@655c52be{SSL,[ssl, http/1.1]}{127.0.0.1:35199}
[junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@190ae2b1{SSL,[ssl, http/1.1]}{127.0.0.1:35307}
[junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ] o.e.j.s.Server Started @2368674ms
[junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ] o.e.j.s.Server Started @2368674ms
[junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35199}
[junit4] 2> 2357427 INFO (jetty-launcher-2652-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35307}
[junit4] 2> 2357427 ERROR (jetty-launcher-2652-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2357427 ERROR (jetty-launcher-2652-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T01:06:00.681Z
[junit4] 2> 2357428 INFO (jetty-launcher-2652-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T01:06:00.681Z
[junit4] 2> 2357435 INFO (zkConnectionManagerCallback-2658-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357436 INFO (jetty-launcher-2652-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2357440 INFO (zkConnectionManagerCallback-2659-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357441 INFO (jetty-launcher-2652-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2357445 INFO (jetty-launcher-2652-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
[junit4] 2> 2357445 INFO (jetty-launcher-2652-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
[junit4] 2> 2357446 INFO (zkConnectionManagerCallback-2663-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357452 INFO (zkConnectionManagerCallback-2667-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357453 INFO (zkConnectionManagerCallback-2669-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357453 INFO (zkConnectionManagerCallback-2671-thread-1-processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357519 INFO (jetty-launcher-2652-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2357519 INFO (jetty-launcher-2652-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:35199_solr
[junit4] 2> 2357520 INFO (jetty-launcher-2652-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.c.Overseer Overseer (id=99430430184177669-127.0.0.1:35199_solr-n_0000000000) starting
[junit4] 2> 2357521 INFO (jetty-launcher-2652-thread-2) [n:127.0.0.1:35307_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2357521 INFO (jetty-launcher-2652-thread-2) [n:127.0.0.1:35307_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35307_solr
[junit4] 2> 2357534 INFO (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2357535 INFO (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2357537 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 2357538 INFO (jetty-launcher-2652-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35199_solr
[junit4] 2> 2357538 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 2357546 INFO (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2357546 INFO (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2357546 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr]
[junit4] 2> 2357546 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 2357546 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 2357547 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2357547 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 2357548 DEBUG (ScheduledTrigger-4813-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2357581 INFO (jetty-launcher-2652-thread-2) [n:127.0.0.1:35307_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35307.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2357582 INFO (jetty-launcher-2652-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35199.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-2) [n:127.0.0.1:35307_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35307.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-2) [n:127.0.0.1:35307_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35307.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35199.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2357589 INFO (jetty-launcher-2652-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35199.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2357590 INFO (jetty-launcher-2652-thread-2) [n:127.0.0.1:35307_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001/node2/.
[junit4] 2> 2357590 INFO (jetty-launcher-2652-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001/node1/.
[junit4] 2> 2357622 INFO (zkConnectionManagerCallback-2677-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357624 INFO (zkConnectionManagerCallback-2681-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357624 INFO (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2357625 INFO (SUITE-TriggerIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46316/solr ready
[junit4] 2> 2357637 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
[junit4] 2> 2357637 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 2357637 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 2357637 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 2357638 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2357638 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2357639 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 2357664 INFO (qtp1081540318-14383) [n:127.0.0.1:35199_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2357665 INFO (qtp702010038-14390) [n:127.0.0.1:35307_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2357666 DEBUG (qtp702010038-14382) [n:127.0.0.1:35307_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2357666 INFO (qtp702010038-14382) [n:127.0.0.1:35307_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=14
[junit4] 2> 2357666 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 2357669 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr]
[junit4] 2> 2357669 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2357669 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 2357669 INFO (qtp1081540318-14387) [n:127.0.0.1:35199_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2357670 INFO (qtp702010038-14384) [n:127.0.0.1:35307_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2357670 INFO (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2357670 DEBUG (qtp702010038-14386) [n:127.0.0.1:35307_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2357671 INFO (qtp702010038-14386) [n:127.0.0.1:35307_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2357678 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 2357678 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr]
[junit4] 2> 2357678 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2357678 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr]
[junit4] 2> 2357679 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2357679 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2357679 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 3
[junit4] 2> 2357679 DEBUG (ScheduledTrigger-4813-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 2357679 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 2357679 DEBUG (ScheduledTrigger-4813-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 2357680 INFO (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2357681 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2357681 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 2357681 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 2357681 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 2357681 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2357682 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2357682 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2357682 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2357682 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@46112453{/solr,null,AVAILABLE}
[junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@435c6f3d{SSL,[ssl, http/1.1]}{127.0.0.1:45825}
[junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server Started @2368930ms
[junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45825}
[junit4] 2> 2357683 ERROR (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2357683 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T01:06:00.936Z
[junit4] 2> 2357684 INFO (zkConnectionManagerCallback-2683-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357685 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2357691 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
[junit4] 2> 2357692 INFO (zkConnectionManagerCallback-2687-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357693 INFO (zkConnectionManagerCallback-2689-thread-1-processing-n:127.0.0.1:45825_solr) [n:127.0.0.1:45825_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2357697 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2357698 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2357699 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45825_solr
[junit4] 2> 2357700 INFO (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2357700 INFO (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2357700 INFO (zkCallback-2688-thread-1-processing-n:127.0.0.1:45825_solr) [n:127.0.0.1:45825_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2357700 INFO (zkCallback-2680-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2357740 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45825.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2357749 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45825.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2357749 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45825.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2357750 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [n:127.0.0.1:45825_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001/node3/.
[junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:45825_solr at time 1642293927204014
[junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:45825_solr] added at times [1642293927204014], now=1642293927367265
[junit4] 2> 2358681 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"5d5a85a25d4aeT3me4vw7c8et15q9500dmer1ag",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":1642293927204014,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1642293927204014],
[junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
[junit4] 2> 2358682 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger2, node_added_trigger1]
[junit4] 2> 2358683 INFO (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2358683 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"5d5a85a25d4aeT3me4vw7c8et15q9500dmer1ag",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":1642293927204014,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1642293927204014],
[junit4] 2> "_enqueue_time_":1642293928447431,
[junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
[junit4] 2> 2358683 INFO (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_added_trigger1
[junit4] 2> 2358684 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 2358684 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 2358684 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=5d5a85a25d4aeT3me4vw7c8et15q9500dmer1ag
[junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-4) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-4) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:45825_solr at time 1642298930755018
[junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:45825_solr] added at times [1642298930755018], now=1642298930823201
[junit4] 2> 2363684 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"5d5a98461f5caT3me4vw7c8et15q9500dmer1ai",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":1642298930755018,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1642298930755018],
[junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
[junit4] 2> 2363685 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger2, node_added_trigger1]
[junit4] 2> 2363685 INFO (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2363685 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"5d5a98461f5caT3me4vw7c8et15q9500dmer1ai",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":1642298930755018,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1642298930755018],
[junit4] 2> "_enqueue_time_":1642298931021681,
[junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
[junit4] 2> 2363685 INFO (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest last action at 1517187961936000000 current time = 1517187966938000000
[junit4] 2> real diff: 5002000000
[junit4] 2> min diff: 4998000000
[junit4] 2> 2363685 INFO (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_added_trigger2
[junit4] 2> 2363686 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 2363686 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 2363686 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 0 ms for event id=5d5a98461f5caT3me4vw7c8et15q9500dmer1ai
[junit4] 2> 2363689 INFO (qtp1081540318-14383) [n:127.0.0.1:35199_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2363691 INFO (qtp702010038-14390) [n:127.0.0.1:35307_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2363705 INFO (qtp2109945748-14443) [n:127.0.0.1:45825_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2363705 DEBUG (qtp702010038-14382) [n:127.0.0.1:35307_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2363706 INFO (qtp702010038-14382) [n:127.0.0.1:35307_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=18
[junit4] 2> 2363706 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 2363706 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
[junit4] 2> 2363706 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2363706 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
[junit4] 2> 2363707 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
[junit4] 2> 2363707 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2363707 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 2363707 INFO (qtp1081540318-14387) [n:127.0.0.1:35199_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2363707 INFO (qtp702010038-14384) [n:127.0.0.1:35307_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2363708 INFO (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2363708 INFO (qtp2109945748-14445) [n:127.0.0.1:45825_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2363708 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 2363708 DEBUG (qtp702010038-14386) [n:127.0.0.1:35307_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2363708 DEBUG (ScheduledTrigger-4813-thread-4) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
[junit4] 2> 2363708 INFO (qtp702010038-14386) [n:127.0.0.1:35307_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
[junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
[junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
[junit4] 2> 2363709 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 2363710 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:35307_solr, 127.0.0.1:35199_solr, 127.0.0.1:45825_solr]
[junit4] 2> 2363710 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 2363711 INFO (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2363711 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 2363712 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@435c6f3d{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 2363712 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1700547441
[junit4] 2> 2363712 DEBUG (ScheduledTrigger-4813-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 3
[junit4] 2> 2363712 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2363712 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4eb5a754: rootName = solr_45825, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3907f8dd
[junit4] 2> 2363717 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2363717 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@103306db: rootName = solr_45825, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@dc0593e
[junit4] 2> 2363721 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2363721 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@23607bff: rootName = solr_45825, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@5503b61b
[junit4] 2> 2363722 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45825_solr
[junit4] 2> 2363723 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 2363723 INFO (zkCallback-2688-thread-1-processing-n:127.0.0.1:45825_solr) [n:127.0.0.1:45825_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2363723 INFO (zkCallback-2680-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2363723 INFO (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2363723 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2363723 INFO (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2363724 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@46112453{/solr,null,UNAVAILABLE}
[junit4] 2> 2363724 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2364709 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
[junit4] 2> 2364709 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:45825_solr
[junit4] 2> 2364710 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:45825_solr]
[junit4] 2> 2364710 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"5d5a9c17d5001T3me4vw7c8et15q9500dmer1ak",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":1642299955957761,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1642299955957761],
[junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
[junit4] 2> 2364710 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 2364710 INFO (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2364710 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"5d5a9c17d5001T3me4vw7c8et15q9500dmer1ak",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":1642299955957761,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1642299955957761],
[junit4] 2> "_enqueue_time_":1642299956256878,
[junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
[junit4] 2> 2364711 INFO (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_lost_trigger1
[junit4] 2> 2364711 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 5000ms
[junit4] 2> 2364711 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 2364711 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 5000ms
[junit4] 2> 2364711 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 2364711 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 0 ms for event id=5d5a9c17d5001T3me4vw7c8et15q9500dmer1ak
[junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-2) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 2
[junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-4) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-1) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-4) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 2369712 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 2
[junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:45825_solr
[junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:45825_solr]
[junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"5d5aaebb1d434T3me4vw7c8et15q9500dmer1am",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":1642304959009844,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1642304959009844],
[junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
[junit4] 2> 2369713 DEBUG (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 2369713 INFO (ScheduledTrigger-4813-thread-3) [n:127.0.0.1:35199_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2369713 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"5d5aaebb1d434T3me4vw7c8et15q9500dmer1am",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":1642304959009844,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1642304959009844],
[junit4] 2> "_enqueue_time_":1642304959242817,
[junit4] 2> "nodeNames":["127.0.0.1:45825_solr"]}}
[junit4] 2> 2369713 INFO (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest last action at 1517187967964000000 current time = 1517187972966000000
[junit4] 2> real diff: 5002000000
[junit4] 2> min diff: 4998000000
[junit4] 2> 2369713 INFO (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_lost_trigger2
[junit4] 2> 2369713 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[DA442618305C7D2C]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
[junit4] 2> 2369717 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 5000ms
[junit4] 2> 2369717 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 2369717 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 5000ms
[junit4] 2> 2369717 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 2369718 DEBUG (AutoscalingActionExecutor-4814-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 4 ms for event id=5d5aaebb1d434T3me4vw7c8et15q9500dmer1am
[junit4] 2> 2369725 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
[junit4] 2> 2369725 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 7
[junit4] 2> 2369725 DEBUG (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 2369725 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 2369728 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2369728 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2369728 DEBUG (OverseerAutoScalingTriggerThread-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 2369729 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2369729 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2369729 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2369729 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2369729 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@62c45afd{/solr,null,AVAILABLE}
[junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1472c7a9{SSL,[ssl, http/1.1]}{127.0.0.1:46816}
[junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server Started @2380977ms
[junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46816}
[junit4] 2> 2369730 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2369730 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T01:06:12.983Z
[junit4] 2> 2369731 INFO (zkConnectionManagerCallback-2693-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2369732 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2369738 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
[junit4] 2> 2369739 INFO (zkConnectionManagerCallback-2697-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2369740 INFO (zkConnectionManagerCallback-2699-thread-1-processing-n:127.0.0.1:46816_solr) [n:127.0.0.1:46816_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2369744 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:46816_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2369745 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:46816_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2369746 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:46816_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46816_solr
[junit4] 2> 2369746 INFO (zkCallback-2680-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2369746 INFO (zkCallback-2668-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2369746 INFO (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2369754 INFO (zkCallback-2698-thread-1-processing-n:127.0.0.1:46816_solr) [n:127.0.0.1:46816_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2369789 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:46816_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46816.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2369797 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:46816_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46816.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2369797 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:46816_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46816.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2369798 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:46816_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001/node4/.
[junit4] 2> 2369830 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2369830 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2369830 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2369831 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2369831 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4bf22c59{/solr,null,AVAILABLE}
[junit4] 2> 2369831 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@15c0228a{SSL,[ssl, http/1.1]}{127.0.0.1:42276}
[junit4] 2> 2369831 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server Started @2381078ms
[junit4] 2> 2369831 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42276}
[junit4] 2> 2369832 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2369832 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2369832 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2369832 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2369832 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T01:06:13.085Z
[junit4] 2> 2369833 INFO (zkConnectionManagerCallback-2703-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2369833 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2369839 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
[junit4] 2> 2369840 INFO (zkConnectionManagerCallback-2707-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2369842 INFO (zkConnectionManagerCallback-2709-thread-1-processing-n:127.0.0.1:42276_solr) [n:127.0.0.1:42276_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2369845 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:42276_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2369847 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:42276_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2369848 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:42276_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42276_solr
[junit4] 2> 2369849 INFO (zkCallback-2698-thread-1-processing-n:127.0.0.1:46816_solr) [n:127.0.0.1:46816_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2369849 INFO (zkCallback-2680-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2369849 INFO (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2369849 INFO (zkCallback-2668-thread-2-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2369849 INFO (zkCallback-2708-thread-1-processing-n:127.0.0.1:42276_solr) [n:127.0.0.1:42276_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2369882 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:42276_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42276.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2369891 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:42276_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42276.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2369891 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:42276_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42276.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2369892 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:42276_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001/node5/.
[junit4] 2> 2369931 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T09:27:37+12:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2369932 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2369932 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2369932 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2369932 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@29cffbc8{/solr,null,AVAILABLE}
[junit4] 2> 2369933 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6d0d5d58{SSL,[ssl, http/1.1]}{127.0.0.1:37399}
[junit4] 2> 2369933 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.e.j.s.Server Started @2381180ms
[junit4] 2> 2369933 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37399}
[junit4] 2> 2369933 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2369933 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2369933 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2369933 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2369934 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-29T01:06:13.187Z
[junit4] 2> 2369935 INFO (zkConnectionManagerCallback-2713-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2369935 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2369941 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46316/solr
[junit4] 2> 2369944 INFO (zkConnectionManagerCallback-2717-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2369946 INFO (zkConnectionManagerCallback-2719-thread-1-processing-n:127.0.0.1:37399_solr) [n:127.0.0.1:37399_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2369950 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:37399_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2369951 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:37399_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2369953 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:37399_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37399_solr
[junit4] 2> 2369953 INFO (zkCallback-2698-thread-1-processing-n:127.0.0.1:46816_solr) [n:127.0.0.1:46816_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2369963 INFO (zkCallback-2668-thread-2-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2369963 INFO (zkCallback-2680-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2369964 INFO (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2369964 INFO (zkCallback-2708-thread-1-processing-n:127.0.0.1:42276_solr) [n:127.0.0.1:42276_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2369964 INFO (zkCallback-2718-thread-1-processing-n:127.0.0.1:37399_solr) [n:127.0.0.1:37399_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2369985 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:37399_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37399.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2369994 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:37399_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37399.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2369994 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:37399_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37399.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6c7a445
[junit4] 2> 2369995 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[DA442618305C7D2C]) [n:127.0.0.1:37399_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_DA442618305C7D2C-001/tempDir-001/node6/.
[junit4] 2> 2370042 INFO (qtp1081540318-14389) [n:127.0.0.1:35199_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=collection1&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2370044 INFO (OverseerThreadFactory-4815-thread-1-processing-n:127.0.0.1:35199_solr) [n:127.0.0.1:35199_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 2370148 INFO (OverseerStateUpdate-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:46816/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2370150 INFO (OverseerStateUpdate-99430430184177669-127.0.0.1:35199_solr-n_0000000000) [n:127.0.0.1:35199_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:35307/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2370354 INFO (qtp702010038-14386) [n:127.0.0.1:35307_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n2&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2370354 INFO (qtp702010038-14386) [n:127.0.0.1:35307_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2370370 INFO (qtp769574592-14471) [n:127.0.0.1:46816_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2370370 INFO (qtp769574592-14471) [n:127.0.0.1:46816_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2370479 INFO (zkCallback-2698-thread-1-processing-n:127.0.0.1:46816_solr) [n:127.0.0.1:46816_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2370479 INFO (zkCallback-2670-thread-1-processing-n:127.0.0.1:35307_solr) [n:127.0.0.1:35307_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2370479 INFO (zkCallback-2698-thread-2-processing-n:127.0.0.1:46816_solr) [n:127.0.0.1:46816_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[j
[...truncated too long message...]
2-thread-1) [ ] o.a.s.c.Overseer Overseer (id=99430488595562501-127.0.0.1:37832_solr-n_0000000000) closing
[junit4] 2> 3318679 INFO (OverseerStateUpdate-99430488595562501-127.0.0.1:37832_solr-n_0000000000) [n:127.0.0.1:37832_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37832_solr
[junit4] 2> 3318698 WARN (OverseerAutoScalingTriggerThread-99430488595562501-127.0.0.1:37832_solr-n_0000000000) [n:127.0.0.1:37832_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3318699 INFO (jetty-closer-3912-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 3318700 INFO (zkCallback-3928-thread-1-processing-n:127.0.0.1:36457_solr) [n:127.0.0.1:36457_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 3318700 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1613f80a8950006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3318700 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1613f80a8950005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3320200 INFO (jetty-closer-3912-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1b89adfc{/solr,null,UNAVAILABLE}
[junit4] 2> 3320201 INFO (jetty-closer-3912-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3320204 WARN (zkCallback-3927-thread-3-processing-n:127.0.0.1:37832_solr) [n:127.0.0.1:37832_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 3320213 INFO (jetty-closer-3912-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@f220276{/solr,null,UNAVAILABLE}
[junit4] 2> 3320213 INFO (jetty-closer-3912-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3320225 ERROR (SUITE-AliasIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3320225 INFO (SUITE-AliasIntegrationTest-seed#[DA442618305C7D2C]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35528 35528
[junit4] 2> 3320507 INFO (Thread-5633) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35528 35528
[junit4] 2> 3320508 WARN (Thread-5633) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 14 /solr/configs/conf
[junit4] 2> 9 /solr/aliases.json
[junit4] 2> 9 /solr/clusterprops.json
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 86 /solr/aliases.json
[junit4] 2> 44 /solr/collections/collection1meta/state.json
[junit4] 2> 24 /solr/collections/collection2meta/state.json
[junit4] 2> 11 /solr/collections/collection_one/state.json
[junit4] 2> 11 /solr/collections/collection1/state.json
[junit4] 2> 11 /solr/collections/collection_old/state.json
[junit4] 2> 11 /solr/collections/testErrorChecks-collection/state.json
[junit4] 2> 9 /solr/clusterstate.json
[junit4] 2> 6 /solr/collections/collection_new/state.json
[junit4] 2> 6 /solr/collections/collection_two/state.json
[junit4] 2> 6 /solr/collections/collection2/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 73 /solr/collections
[junit4] 2> 9 /solr/live_nodes
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.AliasIntegrationTest_DA442618305C7D2C-001
[junit4] 2> Jan 29, 2018 1:22:03 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=true): {}, locale=bg, timezone=Australia/ACT
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=66368416,total=524288000
[junit4] 2> NOTE: All tests run in this JVM: [TestStressUserVersions, TestCursorMarkWithoutUniqueKey, CreateRoutedAliasTest, UpdateRequestProcessorFactoryTest, ExitableDirectoryReaderTest, TestJsonFacets, TestDistribStateManager, TestBlobHandler, TestCryptoKeys, FileUtilsTest, TestTrie, TestManagedSchemaThreadSafety, TestReplicaProperties, TestSolrCloudWithKerberosAlt, TestSearchPerf, TestRecoveryHdfs, TestCloudManagedSchema, DistributedExpandComponentTest, TestLegacyNumericRangeQueryBuilder, TestTolerantUpdateProcessorCloud, TestSubQueryTransformer, BlobRepositoryCloudTest, SolrMetricReporterTest, FieldMutatingUpdateProcessorTest, HdfsRecoverLeaseTest, CursorMarkTest, QueryEqualityTest, TestCloudNestedDocsSort, BasicFunctionalityTest, BasicZkTest, SolrInfoBeanTest, DirectUpdateHandlerOptimizeTest, CollectionStateFormat2Test, TestDistributedGrouping, TestInPlaceUpdatesStandalone, ShardRoutingCustomTest, TestGeoJSONResponseWriter, TermsComponentTest, TestBM25SimilarityFactory, TestNumericTokenStream, ExecutePlanActionTest, TestCollectionsAPIViaSolrCloudCluster, RecoveryAfterSoftCommitTest, QueryParsingTest, DocValuesNotIndexedTest, ZkControllerTest, QueryElevationComponentTest, ManagedSchemaRoundRobinCloudTest, TestBackupRepositoryFactory, SimpleMLTQParserTest, SSLMigrationTest, ParsingFieldUpdateProcessorsTest, TestLRUCache, TestCloudRecovery, TestCollapseQParserPlugin, TestSweetSpotSimilarityFactory, AlternateDirectoryTest, TestFastLRUCache, ConcurrentDeleteAndCreateCollectionTest, ConjunctionSolrSpellCheckerTest, TestAuthenticationFramework, TestDynamicFieldResource, RequiredFieldsTest, TestSimpleQParserPlugin, TestCustomStream, TestSSLRandomization, TemplateUpdateProcessorTest, TestSolr4Spatial, TestPerFieldSimilarity, TestMiniSolrCloudClusterSSL, TestHalfAndHalfDocValues, TestUnifiedSolrHighlighter, TestSortByMinMaxFunction, AnalysisErrorHandlingTest, XsltUpdateRequestHandlerTest, DistributedFacetPivotLongTailTest, TestRecovery, TestScoreJoinQPScore, SuggestComponentContextFilterQueryTest, TestExceedMaxTermLength, TestSimpleTrackingShardHandler, DistributedFacetPivotWhiteBoxTest, BlockJoinFacetRandomTest, TestCustomSort, TlogReplayBufferedWhileIndexingTest, SchemaVersionSpecificBehaviorTest, ScriptEngineTest, BasicAuthStandaloneTest, ShardSplitTest, SoftAutoCommitTest, MigrateRouteKeyTest, PathHierarchyTokenizerFactoryTest, TestStressVersions, TestBlendedInfixSuggestions, TestZkChroot, HdfsUnloadDistributedZkTest, FullHLLTest, TestDownShardTolerantSearch, PolyFieldTest, TestRTimerTree, TestPartialUpdateDeduplication, TestInfoStreamLogging, PeerSyncTest, SecurityConfHandlerTest, TriggerIntegrationTest, SystemInfoHandlerTest, RegexBytesRefFilterTest, TestSchemaResource, PrimUtilsTest, TestExpandComponent, TestBulkSchemaAPI, ConfigureRecoveryStrategyTest, SparseHLLTest, TestSystemIdResolver, WrapperMergePolicyFactoryTest, TestPKIAuthenticationPlugin, LeaderElectionContextKeyTest, TestPolicyCloud, DeleteStatusTest, TestNonDefinedSimilarityFactory, TestSolrCoreParser, TestPullReplica, ResourceLoaderTest, CdcrReplicationDistributedZkTest, TestStressInPlaceUpdates, TestCoreAdminApis, V2ApiIntegrationTest, DistributedIntervalFacetingTest, TestMacros, SyncSliceTest, OverseerTest, RecoveryZkTest, TestRandomFaceting, LeaderElectionTest, TestFaceting, TestHashPartitioner, TestStressReorder, TestReload, TestRangeQuery, TestFoldingMultitermQuery, SuggesterTSTTest, SuggesterTest, TestStressLucene, NoCacheHeaderTest, TestUpdate, TestWordDelimiterFilterFactory, StatelessScriptUpdateProcessorFactoryTest, SortByFunctionTest, TestRemoteStreaming, DocValuesMultiTest, DistanceFunctionTest, DebugComponentTest, CacheHeaderTest, TestSurroundQueryParser, DisMaxRequestHandlerTest, DirectSolrSpellCheckerTest, PrimitiveFieldTypeTest, DocumentBuilderTest, TestIndexingPerformance, FastVectorHighlighterTest, LoggingHandlerTest, JsonLoaderTest, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, TestComponentsName, SearchHandlerTest, OutputWriterTest, DirectSolrConnectionTest, TestCharFilters, SynonymTokenizerTest, TestDFRSimilarityFactory, TestIBSimilarityFactory, TestFastWriter, URLClassifyProcessorTest, DateMathParserTest, PreAnalyzedFieldTest, DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, ClusterStateTest, TestSolrJ, SliceStateTest, CircularListTest, DistributedMLTComponentTest, TestRTGBase, SolrTestCaseJ4Test, TestCrossCoreJoin, TestTolerantSearch, TestLuceneIndexBackCompat, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, ActionThrottleTest, AliasIntegrationTest]
[junit4] Completed [513/769 (2!)] on J1 in 72.23s, 8 tests, 1 error <<< FAILURES!
[...truncated 48970 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 338 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/338/
3 tests failed.
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Could not find collection:collection2
Stack Trace:
java.lang.AssertionError: Could not find collection:collection2
at __randomizedtesting.SeedInfo.seed([1F42771EB703FF04:971648C419FF92FC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:915)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:612)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue: 1) Thread[id=15126, name=sdqtest--1623-thread-1, state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:194) at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:167) at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.lambda$testDistributedQueueBlocking$1(TestSimDistributedQueue.java:107) at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue$$Lambda$803/62862484.call(Unknown Source) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$9/1527288253.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue:
1) Thread[id=15126, name=sdqtest--1623-thread-1, state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:194)
at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:167)
at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.lambda$testDistributedQueueBlocking$1(TestSimDistributedQueue.java:107)
at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue$$Lambda$803/62862484.call(Unknown Source)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$9/1527288253.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([1F42771EB703FF04]:0)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueueBlocking
Error Message:
Stack Trace:
java.util.concurrent.TimeoutException
at __randomizedtesting.SeedInfo.seed([1F42771EB703FF04:5AE80567F3514370]:0)
at java.util.concurrent.FutureTask.get(FutureTask.java:205)
at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.testDistributedQueueBlocking(TestSimDistributedQueue.java:116)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12940 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/init-core-data-001
[junit4] 2> 2332570 WARN (SUITE-FullSolrCloudDistribCmdsTest-seed#[1F42771EB703FF04]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=10 numCloses=10
[junit4] 2> 2332570 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[1F42771EB703FF04]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2332572 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[1F42771EB703FF04]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 2332572 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[1F42771EB703FF04]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2332572 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[1F42771EB703FF04]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 2332578 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2332578 INFO (Thread-7028) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2332578 INFO (Thread-7028) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2332588 ERROR (Thread-7028) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2332678 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.ZkTestServer start zk server on port:45039
[junit4] 2> 2332683 INFO (zkConnectionManagerCallback-3287-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2332688 INFO (zkConnectionManagerCallback-3289-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2332691 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2332692 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 2332693 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2332694 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2332695 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2332695 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2332696 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2332697 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2332697 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2332698 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2332699 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2332700 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 2332773 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2332787 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2332787 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2332787 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2332787 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4155ab38{/,null,AVAILABLE}
[junit4] 2> 2332796 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1db4bd90{HTTP/1.1,[http/1.1]}{127.0.0.1:46079}
[junit4] 2> 2332797 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.Server Started @2342850ms
[junit4] 2> 2332797 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/tempDir-001/control/data, hostContext=/, hostPort=46079, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/control-001/cores}
[junit4] 2> 2332797 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2332797 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2332797 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2332797 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2332797 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-28T18:51:28.411Z
[junit4] 2> 2332798 INFO (zkConnectionManagerCallback-3291-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2332799 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2332799 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/control-001/solr.xml
[junit4] 2> 2332802 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2332802 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2332803 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2332812 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45039/solr
[junit4] 2> 2332813 INFO (zkConnectionManagerCallback-3295-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2332829 INFO (zkConnectionManagerCallback-3297-thread-1-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2332877 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:46079_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2332877 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:46079_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46079_
[junit4] 2> 2332878 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:46079_ ] o.a.s.c.Overseer Overseer (id=99428957434216452-127.0.0.1:46079_-n_0000000000) starting
[junit4] 2> 2332882 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:46079_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46079_
[junit4] 2> 2332897 INFO (OverseerStateUpdate-99428957434216452-127.0.0.1:46079_-n_0000000000) [n:127.0.0.1:46079_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2332924 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:46079_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2332932 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:46079_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2332932 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:46079_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2332933 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:46079_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/control-001/cores
[junit4] 2> 2332948 INFO (zkConnectionManagerCallback-3303-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2332949 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2332950 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45039/solr ready
[junit4] 2> 2332951 INFO (qtp417451036-19237) [n:127.0.0.1:46079_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:46079_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2332954 INFO (OverseerThreadFactory-5034-thread-1-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 2333058 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2333058 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2333163 INFO (zkCallback-3296-thread-1-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2334072 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2334103 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 2334171 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2334181 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 2334181 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2334181 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2334182 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 2334184 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=13, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2334186 WARN (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2334217 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2334217 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2334218 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2334218 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2334219 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=34.283203125, floorSegmentMB=0.8310546875, forceMergeDeletesPctAllowed=22.95865994223088, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5614593543168842
[junit4] 2> 2334219 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@68c0fb0d[control_collection_shard1_replica_n1] main]
[junit4] 2> 2334220 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2334220 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2334221 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2334221 INFO (searcherExecutor-5037-thread-1-processing-n:127.0.0.1:46079_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@68c0fb0d[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2334221 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590863320669224960
[junit4] 2> 2334225 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2334225 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2334225 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46079/control_collection_shard1_replica_n1/
[junit4] 2> 2334225 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2334225 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:46079/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 2334225 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2334227 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46079/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 2334328 INFO (zkCallback-3296-thread-2-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2334378 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2334379 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1321
[junit4] 2> 2334384 INFO (qtp417451036-19237) [n:127.0.0.1:46079_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2334480 INFO (zkCallback-3296-thread-2-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2334956 INFO (OverseerCollectionConfigSetProcessor-99428957434216452-127.0.0.1:46079_-n_0000000000) [n:127.0.0.1:46079_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2335384 INFO (qtp417451036-19237) [n:127.0.0.1:46079_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:46079_&wt=javabin&version=2} status=0 QTime=2433
[junit4] 2> 2335387 INFO (zkConnectionManagerCallback-3307-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2335388 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2335388 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45039/solr ready
[junit4] 2> 2335388 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2335389 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2335390 INFO (OverseerThreadFactory-5034-thread-2-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 2335391 WARN (OverseerThreadFactory-5034-thread-2-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 2335596 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2335596 INFO (qtp417451036-19241) [n:127.0.0.1:46079_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=207
[junit4] 2> 2335678 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-1-001 of type NRT
[junit4] 2> 2335678 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2335680 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2335680 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2335680 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2335680 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3127662c{/,null,AVAILABLE}
[junit4] 2> 2335681 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@10bfc278{HTTP/1.1,[http/1.1]}{127.0.0.1:33972}
[junit4] 2> 2335681 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.Server Started @2345734ms
[junit4] 2> 2335681 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=33972, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-1-001/cores}
[junit4] 2> 2335681 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2335681 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2335681 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2335681 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2335681 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-28T18:51:31.295Z
[junit4] 2> 2335683 INFO (zkConnectionManagerCallback-3309-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2335684 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2335684 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-1-001/solr.xml
[junit4] 2> 2335697 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2335697 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2335698 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2335700 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45039/solr
[junit4] 2> 2335701 INFO (zkConnectionManagerCallback-3313-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2335704 INFO (zkConnectionManagerCallback-3315-thread-1-processing-n:127.0.0.1:33972_) [n:127.0.0.1:33972_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2335708 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:33972_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2335710 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:33972_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2335711 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:33972_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33972_
[junit4] 2> 2335711 INFO (zkCallback-3296-thread-2-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2335723 INFO (zkCallback-3306-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2335724 INFO (zkCallback-3314-thread-1-processing-n:127.0.0.1:33972_) [n:127.0.0.1:33972_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2335763 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:33972_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2335772 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:33972_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2335772 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:33972_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2335773 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:33972_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-1-001/cores
[junit4] 2> 2335795 INFO (qtp1286338733-19285) [n:127.0.0.1:33972_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:33972_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2335797 INFO (OverseerCollectionConfigSetProcessor-99428957434216452-127.0.0.1:46079_-n_0000000000) [n:127.0.0.1:46079_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2335798 INFO (OverseerThreadFactory-5034-thread-3-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:33972_ for creating new replica
[junit4] 2> 2335799 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2335800 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2335904 INFO (zkCallback-3314-thread-1-processing-n:127.0.0.1:33972_) [n:127.0.0.1:33972_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2336816 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2336826 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema [collection1_shard2_replica_n61] Schema name=test
[junit4] 2> 2336902 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2336912 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n61' using configuration from collection collection1, trusted=true
[junit4] 2> 2336912 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n61' (registry 'solr.core.collection1.shard2.replica_n61') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2336912 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2336912 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [[collection1_shard2_replica_n61] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-1-001/cores/collection1_shard2_replica_n61], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-1-001/cores/collection1_shard2_replica_n61/data/]
[junit4] 2> 2336915 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=13, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2336916 WARN (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2336946 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2336946 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2336947 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2336947 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2336948 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=34.283203125, floorSegmentMB=0.8310546875, forceMergeDeletesPctAllowed=22.95865994223088, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5614593543168842
[junit4] 2> 2336948 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.SolrIndexSearcher Opening [Searcher@70227320[collection1_shard2_replica_n61] main]
[junit4] 2> 2336949 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2336949 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2336950 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2336951 INFO (searcherExecutor-5048-thread-1-processing-n:127.0.0.1:33972_ x:collection1_shard2_replica_n61 s:shard2 c:collection1) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [collection1_shard2_replica_n61] Registered new searcher Searcher@70227320[collection1_shard2_replica_n61] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2336951 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590863323531837440
[junit4] 2> 2336954 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2336954 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2336954 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:33972/collection1_shard2_replica_n61/
[junit4] 2> 2336954 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2336955 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy http://127.0.0.1:33972/collection1_shard2_replica_n61/ has no replicas
[junit4] 2> 2336955 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2336957 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:33972/collection1_shard2_replica_n61/ shard2
[junit4] 2> 2337058 INFO (zkCallback-3314-thread-1-processing-n:127.0.0.1:33972_) [n:127.0.0.1:33972_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2337107 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2337109 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1309
[junit4] 2> 2337110 INFO (qtp1286338733-19285) [n:127.0.0.1:33972_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:33972_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1314
[junit4] 2> 2337183 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-2-001 of type NRT
[junit4] 2> 2337183 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2337184 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2337184 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2337184 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2337184 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1b82d47f{/,null,AVAILABLE}
[junit4] 2> 2337185 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6a3b70c9{HTTP/1.1,[http/1.1]}{127.0.0.1:40398}
[junit4] 2> 2337185 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.Server Started @2347238ms
[junit4] 2> 2337185 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/, hostPort=40398, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-2-001/cores}
[junit4] 2> 2337185 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2337187 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2337187 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2337187 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2337187 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-28T18:51:32.801Z
[junit4] 2> 2337189 INFO (zkConnectionManagerCallback-3319-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2337189 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2337189 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-2-001/solr.xml
[junit4] 2> 2337192 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2337192 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2337193 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2337196 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45039/solr
[junit4] 2> 2337196 INFO (zkConnectionManagerCallback-3323-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2337200 INFO (zkConnectionManagerCallback-3325-thread-1-processing-n:127.0.0.1:40398_) [n:127.0.0.1:40398_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2337203 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:40398_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2337205 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:40398_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2337206 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:40398_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40398_
[junit4] 2> 2337207 INFO (zkCallback-3306-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2337207 INFO (zkCallback-3314-thread-1-processing-n:127.0.0.1:33972_) [n:127.0.0.1:33972_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2337207 INFO (zkCallback-3296-thread-2-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2337223 INFO (zkCallback-3324-thread-1-processing-n:127.0.0.1:40398_) [n:127.0.0.1:40398_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2337242 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:40398_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2337251 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:40398_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2337251 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:40398_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2337252 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:40398_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-2-001/cores
[junit4] 2> 2337281 INFO (qtp1286338733-19287) [n:127.0.0.1:33972_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:40398_&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2337283 INFO (OverseerCollectionConfigSetProcessor-99428957434216452-127.0.0.1:46079_-n_0000000000) [n:127.0.0.1:46079_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2337283 INFO (OverseerThreadFactory-5034-thread-4-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:40398_ for creating new replica
[junit4] 2> 2337285 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n63&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2337286 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2337388 INFO (zkCallback-3314-thread-1-processing-n:127.0.0.1:33972_) [n:127.0.0.1:33972_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2337388 INFO (zkCallback-3324-thread-1-processing-n:127.0.0.1:40398_) [n:127.0.0.1:40398_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2338298 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2338308 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema [collection1_shard3_replica_n63] Schema name=test
[junit4] 2> 2338385 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2338394 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard3_replica_n63' using configuration from collection collection1, trusted=true
[junit4] 2> 2338394 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard3.replica_n63' (registry 'solr.core.collection1.shard3.replica_n63') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2338394 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2338395 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore [[collection1_shard3_replica_n63] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-2-001/cores/collection1_shard3_replica_n63], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-2-001/cores/collection1_shard3_replica_n63/data/]
[junit4] 2> 2338396 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=13, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2338398 WARN (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2338428 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2338428 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2338429 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2338429 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2338430 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=34.283203125, floorSegmentMB=0.8310546875, forceMergeDeletesPctAllowed=22.95865994223088, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5614593543168842
[junit4] 2> 2338431 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.SolrIndexSearcher Opening [Searcher@508fca4d[collection1_shard3_replica_n63] main]
[junit4] 2> 2338431 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2338432 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2338432 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2338432 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590863325084778496
[junit4] 2> 2338438 INFO (searcherExecutor-5059-thread-1-processing-n:127.0.0.1:40398_ x:collection1_shard3_replica_n63 s:shard3 c:collection1) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore [collection1_shard3_replica_n63] Registered new searcher Searcher@508fca4d[collection1_shard3_replica_n63] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2338439 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2338439 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2338439 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:40398/collection1_shard3_replica_n63/
[junit4] 2> 2338439 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2338439 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy http://127.0.0.1:40398/collection1_shard3_replica_n63/ has no replicas
[junit4] 2> 2338439 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2338441 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:40398/collection1_shard3_replica_n63/ shard3
[junit4] 2> 2338546 INFO (zkCallback-3324-thread-1-processing-n:127.0.0.1:40398_) [n:127.0.0.1:40398_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2338546 INFO (zkCallback-3314-thread-1-processing-n:127.0.0.1:33972_) [n:127.0.0.1:33972_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2338592 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2338593 INFO (qtp598864955-19317) [n:127.0.0.1:40398_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n63&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1307
[junit4] 2> 2338594 INFO (qtp1286338733-19287) [n:127.0.0.1:33972_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:40398_&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2} status=0 QTime=1313
[junit4] 2> 2338667 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-3-001 of type NRT
[junit4] 2> 2338668 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2338668 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2338668 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2338668 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2338669 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2e926047{/,null,AVAILABLE}
[junit4] 2> 2338669 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@58fcb9df{HTTP/1.1,[http/1.1]}{127.0.0.1:36727}
[junit4] 2> 2338669 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.e.j.s.Server Started @2348723ms
[junit4] 2> 2338669 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/tempDir-001/jetty3, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=36727, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-3-001/cores}
[junit4] 2> 2338669 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2338672 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 2338672 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2338672 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2338672 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-28T18:51:34.286Z
[junit4] 2> 2338673 INFO (zkConnectionManagerCallback-3329-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2338674 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2338674 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-3-001/solr.xml
[junit4] 2> 2338677 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2338677 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2338678 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2338680 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45039/solr
[junit4] 2> 2338680 INFO (zkConnectionManagerCallback-3333-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2338684 INFO (zkConnectionManagerCallback-3335-thread-1-processing-n:127.0.0.1:36727_) [n:127.0.0.1:36727_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2338687 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:36727_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2338688 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:36727_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2338689 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:36727_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36727_
[junit4] 2> 2338690 INFO (zkCallback-3324-thread-1-processing-n:127.0.0.1:40398_) [n:127.0.0.1:40398_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2338690 INFO (zkCallback-3306-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2338690 INFO (zkCallback-3314-thread-1-processing-n:127.0.0.1:33972_) [n:127.0.0.1:33972_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2338690 INFO (zkCallback-3296-thread-2-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2338699 INFO (zkCallback-3334-thread-1-processing-n:127.0.0.1:36727_) [n:127.0.0.1:36727_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2338734 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:36727_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2338743 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:36727_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2338743 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:36727_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2338744 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[1F42771EB703FF04]) [n:127.0.0.1:36727_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-3-001/cores
[junit4] 2> 2338781 INFO (qtp1286338733-19289) [n:127.0.0.1:33972_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:36727_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2338782 INFO (OverseerCollectionConfigSetProcessor-99428957434216452-127.0.0.1:46079_-n_0000000000) [n:127.0.0.1:46079_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2338783 INFO (OverseerThreadFactory-5034-thread-5-processing-n:127.0.0.1:46079_) [n:127.0.0.1:46079_ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:36727_ for creating new replica
[junit4] 2> 2338784 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n65&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2338785 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2338891 INFO (zkCallback-3334-thread-1-processing-n:127.0.0.1:36727_) [n:127.0.0.1:36727_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2338891 INFO (zkCallback-3324-thread-1-processing-n:127.0.0.1:40398_) [n:127.0.0.1:40398_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2338891 INFO (zkCallback-3314-thread-1-processing-n:127.0.0.1:33972_) [n:127.0.0.1:33972_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2339798 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 2339808 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.IndexSchema [collection1_shard1_replica_n65] Schema name=test
[junit4] 2> 2339886 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 2339900 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n65' using configuration from collection collection1, trusted=true
[junit4] 2> 2339901 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n65' (registry 'solr.core.collection1.shard1.replica_n65') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@53c93f4e
[junit4] 2> 2339901 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2339901 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrCore [[collection1_shard1_replica_n65] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-3-001/cores/collection1_shard1_replica_n65], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_1F42771EB703FF04-001/shard-3-001/cores/collection1_shard1_replica_n65/data/]
[junit4] 2> 2339903 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=13, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
[junit4] 2> 2339905 WARN (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2339935 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2339935 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2339936 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2339936 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2339937 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=18, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=34.283203125, floorSegmentMB=0.8310546875, forceMergeDeletesPctAllowed=22.95865994223088, segmentsPerTier=33.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5614593543168842
[junit4] 2> 2339937 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.SolrIndexSearcher Opening [Searcher@7c5b4f77[collection1_shard1_replica_n65] main]
[junit4] 2> 2339938 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2339938 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2339939 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2339940 INFO (searcherExecutor-5070-thread-1-processing-n:127.0.0.1:36727_ x:collection1_shard1_replica_n65 s:shard1 c:collection1) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrCore [collection1_shard1_replica_n65] Registered new searcher Searcher@7c5b4f77[collection1_shard1_replica_n65] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2339940 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590863326666031104
[junit4] 2> 2339943 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2339943 INFO (qtp951148700-19348) [n:127.0.0.1:36727_ c:collection1 s:shard1 x:col
[...truncated too long message...]
urandom
[junit4] 2> 2535923 INFO (TEST-TestGenericDistributedQueue.testDistributedQueue-seed#[1F42771EB703FF04]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDistributedQueue
[junit4] 2> 2538061 INFO (TEST-TestGenericDistributedQueue.testDistributedQueue-seed#[1F42771EB703FF04]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDistributedQueue
[junit4] 2> 2538063 INFO (TEST-TestGenericDistributedQueue.testLocallyOffer-seed#[1F42771EB703FF04]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testLocallyOffer
[junit4] 2> 2538568 INFO (TEST-TestGenericDistributedQueue.testLocallyOffer-seed#[1F42771EB703FF04]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testLocallyOffer
[junit4] 2> 2538569 INFO (TEST-TestGenericDistributedQueue.testDistributedQueueBlocking-seed#[1F42771EB703FF04]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testDistributedQueueBlocking
[junit4] 2> 2541732 INFO (TEST-TestGenericDistributedQueue.testDistributedQueueBlocking-seed#[1F42771EB703FF04]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testDistributedQueueBlocking
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGenericDistributedQueue -Dtests.method=testDistributedQueueBlocking -Dtests.seed=1F42771EB703FF04 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=nn-NO -Dtests.timezone=America/Ensenada -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 3.17s J0 | TestGenericDistributedQueue.testDistributedQueueBlocking <<<
[junit4] > Throwable #1: java.util.concurrent.TimeoutException
[junit4] > at __randomizedtesting.SeedInfo.seed([1F42771EB703FF04:5AE80567F3514370]:0)
[junit4] > at java.util.concurrent.FutureTask.get(FutureTask.java:205)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.testDistributedQueueBlocking(TestSimDistributedQueue.java:116)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2541735 INFO (TEST-TestGenericDistributedQueue.testPeekElements-seed#[1F42771EB703FF04]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testPeekElements
[junit4] 2> 2543312 INFO (TEST-TestGenericDistributedQueue.testPeekElements-seed#[1F42771EB703FF04]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testPeekElements
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestGenericDistributedQueue_1F42771EB703FF04-001
[junit4] 2> Jan 28, 2018 6:54:59 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Jan 28, 2018 6:55:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue:
[junit4] 2> 1) Thread[id=15126, name=sdqtest--1623-thread-1, state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:194)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:167)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.lambda$testDistributedQueueBlocking$1(TestSimDistributedQueue.java:107)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue$$Lambda$803/62862484.call(Unknown Source)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$9/1527288253.run(Unknown Source)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Jan 28, 2018 6:55:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=15126, name=sdqtest--1623-thread-1, state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue]
[junit4] 2> Jan 28, 2018 6:55:19 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: All leaked threads terminated.
[junit4] 2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=13994, maxDocsPerChunk=229, blockSize=9), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=13994, blockSize=9)), sim=RandomSimilarity(queryNorm=true): {}, locale=nn-NO, timezone=America/Ensenada
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=223575360,total=531103744
[junit4] 2> NOTE: All tests run in this JVM: [AssignBackwardCompatibilityTest, TestReRankQParserPlugin, OutputWriterTest, InfixSuggestersTest, DistanceUnitsTest, TestPivotHelperCode, SpatialHeatmapFacetsTest, MergeStrategyTest, StatsReloadRaceTest, TestMergePolicyConfig, TestSchemaNameResource, UninvertDocValuesMergePolicyTest, TestExportWriter, TestOverriddenPrefixQueryForCustomFieldType, TestNumericTerms32, TestConfigSetImmutable, SuggesterWFSTTest, TestExactSharedStatsCache, CoreMergeIndexesAdminHandlerTest, TestJoin, DebugComponentTest, HdfsRestartWhileUpdatingTest, TestManagedStopFilterFactory, ZkCLITest, ClassificationUpdateProcessorTest, CdcrRequestHandlerTest, NodeLostTriggerTest, TestCoreBackup, VersionInfoTest, CloudMLTQParserTest, BasicDistributedZkTest, TestConfig, TestManagedSchemaAPI, QueryResultKeyTest, TestUtilizeNode, BasicAuthIntegrationTest, SmileWriterTest, TestCrossCoreJoin, RAMDirectoryFactoryTest, RequestHandlersTest, StressHdfsTest, SolrPluginUtilsTest, MultiThreadedOCPTest, TestCloudPseudoReturnFields, EchoParamsTest, HighlighterConfigTest, HdfsLockFactoryTest, MoveReplicaHDFSFailoverTest, TestPrepRecovery, TestSha256AuthenticationProvider, CloudExitableDirectoryReaderTest, HLLUtilTest, TestLegacyTerms, TestStandardQParsers, JvmMetricsTest, SimpleFacetsTest, SolrShardReporterTest, TestCollationFieldDocValues, TestOmitPositions, XmlUpdateRequestHandlerTest, DistributedFacetPivotLargeTest, ZkStateWriterTest, SolrRequestParserTest, TimeRoutedAliasUpdateProcessorTest, TestSchemaManager, SynonymTokenizerTest, UpdateParamsTest, FieldAnalysisRequestHandlerTest, TestCloudInspectUtil, ActionThrottleTest, TestTriggerIntegration, ChaosMonkeySafeLeaderWithPullReplicasTest, BadCopyFieldTest, TestCSVLoader, BadIndexSchemaTest, TestStressLucene, TestEmbeddedSolrServerSchemaAPI, TestInitQParser, ChaosMonkeyNothingIsSafeTest, TestFieldResource, CurrencyFieldTypeTest, HdfsNNFailoverTest, RollingRestartTest, TestAddFieldRealTimeGet, StatsComponentTest, TestStressCloudBlindAtomicUpdates, TestGroupingSearch, TestDocBasedVersionConstraints, TestGenericDistributedQueue]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestGenericDistributedQueue -Dtests.seed=1F42771EB703FF04 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=nn-NO -Dtests.timezone=America/Ensenada -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J0 | TestGenericDistributedQueue (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue:
[junit4] > 1) Thread[id=15126, name=sdqtest--1623-thread-1, state=TIMED_WAITING, group=TGRP-TestGenericDistributedQueue]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:194)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.GenericDistributedQueue.peek(GenericDistributedQueue.java:167)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue.lambda$testDistributedQueueBlocking$1(TestSimDistributedQueue.java:107)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestSimDistributedQueue$$Lambda$803/62862484.call(Unknown Source)
[junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] > at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$9/1527288253.run(Unknown Source)
[junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([1F42771EB703FF04]:0)
[junit4] Completed [372/769 (2!)] on J0 in 27.73s, 4 tests, 2 errors <<< FAILURES!
[...truncated 49437 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 337 - Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/337/
5 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd: 1) Thread[id=157, name=qtp1016972771-157, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd:
1) Thread[id=157, name=qtp1016972771-157, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([B254D9F0868961D2]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=157, name=qtp1016972771-157, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=157, name=qtp1016972771-157, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([B254D9F0868961D2]:0)
FAILED: org.apache.solr.cloud.PeerSyncReplicationTest.test
Error Message:
expected:<0> but was:<1>
Stack Trace:
java.lang.AssertionError: expected:<0> but was:<1>
at __randomizedtesting.SeedInfo.seed([4E1384E88A72F977:C647BB32248E948F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:203)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardRoutingTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardRoutingTest: 1) Thread[id=15300, name=qtp1544677836-15300, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardRoutingTest:
1) Thread[id=15300, name=qtp1544677836-15300, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([4E1384E88A72F977]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardRoutingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=15300, name=qtp1544677836-15300, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=15300, name=qtp1544677836-15300, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([4E1384E88A72F977]:0)
Build Log:
[...truncated 11742 lines...]
[junit4] Suite: org.apache.solr.cloud.PeerSyncReplicationTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/init-core-data-001
[junit4] 2> 85349 WARN (SUITE-PeerSyncReplicationTest-seed#[4E1384E88A72F977]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=18 numCloses=18
[junit4] 2> 85350 INFO (SUITE-PeerSyncReplicationTest-seed#[4E1384E88A72F977]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 85351 INFO (SUITE-PeerSyncReplicationTest-seed#[4E1384E88A72F977]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 85351 INFO (SUITE-PeerSyncReplicationTest-seed#[4E1384E88A72F977]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 85351 INFO (SUITE-PeerSyncReplicationTest-seed#[4E1384E88A72F977]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /di/ri
[junit4] 2> 85488 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 85489 INFO (Thread-90) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 85490 INFO (Thread-90) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 85766 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.ZkTestServer start zk server on port:44406
[junit4] 2> 85965 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Exception causing close of session 0x0 due to java.io.IOException: ZooKeeperServer not running
[junit4] 2> 86067 ERROR (Thread-90) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 88079 INFO (zkConnectionManagerCallback-26-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 88170 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1613a4db5500000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 88198 INFO (zkConnectionManagerCallback-28-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 88282 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 88343 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 88345 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 88359 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 88361 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 88376 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 88378 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 88387 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 88389 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 88398 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 88400 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 88435 INFO (zkConnectionManagerCallback-31-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 88495 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1613a4db5500002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 88496 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 88957 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 88966 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 88966 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 88966 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 88967 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5bdf0461{/di/ri,null,AVAILABLE}
[junit4] 2> 88967 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3dd55460{SSL,[ssl, http/1.1]}{127.0.0.1:36724}
[junit4] 2> 88967 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.Server Started @99471ms
[junit4] 2> 88967 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/tempDir-001/control/data, replicaType=NRT, hostContext=/di/ri, hostPort=36724, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/control-001/cores}
[junit4] 2> 88968 ERROR (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 88968 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 88968 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 88968 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 88968 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-28T01:07:09.905Z
[junit4] 2> 89010 INFO (zkConnectionManagerCallback-33-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 89055 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 89055 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/control-001/solr.xml
[junit4] 2> 89088 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 89088 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 89089 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 89176 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44406/solr
[junit4] 2> 89318 INFO (zkConnectionManagerCallback-37-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 89511 INFO (zkConnectionManagerCallback-39-thread-1-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 90539 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 90541 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36724_di%2Fri
[junit4] 2> 90555 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.Overseer Overseer (id=99424772207673349-127.0.0.1:36724_di%2Fri-n_0000000000) starting
[junit4] 2> 90968 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36724_di%2Fri
[junit4] 2> 91133 INFO (OverseerStateUpdate-99424772207673349-127.0.0.1:36724_di%2Fri-n_0000000000) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 92131 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:36724_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 92460 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:36724_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 92460 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:36724_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 92528 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/control-001/cores
[junit4] 2> 92961 INFO (zkConnectionManagerCallback-45-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 93192 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 93316 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44406/solr ready
[junit4] 2> 93858 INFO (qtp524226505-248) [n:127.0.0.1:36724_di%2Fri ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:36724_di%252Fri&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 94111 INFO (OverseerThreadFactory-125-thread-1-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.a.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 94959 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 94960 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 95509 INFO (zkCallback-38-thread-1-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 96160 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 96349 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 97309 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 97631 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 97644 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 97682 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 97682 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 97790 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=37, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 97926 WARN (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 98430 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 98430 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 98431 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 98431 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 98433 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1727434677, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 98489 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@527cccdd[control_collection_shard1_replica_n1] main]
[junit4] 2> 98525 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 98525 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 98526 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 98598 INFO (searcherExecutor-128-thread-1-processing-n:127.0.0.1:36724_di%2Fri x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@527cccdd[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 98598 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590796368777052160
[junit4] 2> 98760 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 98761 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 98761 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:36724/di/ri/control_collection_shard1_replica_n1/
[junit4] 2> 98794 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 98794 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:36724/di/ri/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 98794 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 98905 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:36724/di/ri/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 98981 INFO (zkCallback-38-thread-2-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 99139 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 99229 INFO (qtp524226505-242) [n:127.0.0.1:36724_di%2Fri c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=4277
[junit4] 2> 99457 INFO (zkCallback-38-thread-1-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 99773 INFO (qtp524226505-248) [n:127.0.0.1:36724_di%2Fri ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 99773 INFO (qtp524226505-248) [n:127.0.0.1:36724_di%2Fri ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:36724_di%252Fri&wt=javabin&version=2} status=0 QTime=5916
[junit4] 2> 100015 INFO (zkConnectionManagerCallback-49-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 100019 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 100020 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44406/solr ready
[junit4] 2> 100021 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 100068 INFO (qtp524226505-246) [n:127.0.0.1:36724_di%2Fri ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 100091 INFO (OverseerThreadFactory-125-thread-2-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 100139 INFO (OverseerCollectionConfigSetProcessor-99424772207673349-127.0.0.1:36724_di%2Fri-n_0000000000) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 100196 WARN (OverseerThreadFactory-125-thread-2-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.a.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 100840 INFO (qtp524226505-246) [n:127.0.0.1:36724_di%2Fri ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 100841 INFO (qtp524226505-246) [n:127.0.0.1:36724_di%2Fri ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=772
[junit4] 2> 101584 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-1-001 of type NRT
[junit4] 2> 101603 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 101687 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 101687 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 101687 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 101703 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35e31226{/di/ri,null,AVAILABLE}
[junit4] 2> 101704 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@22e3e066{SSL,[ssl, http/1.1]}{127.0.0.1:45004}
[junit4] 2> 101704 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.Server Started @112207ms
[junit4] 2> 101704 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/tempDir-001/jetty1, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/di/ri, hostPort=45004, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-1-001/cores}
[junit4] 2> 101704 ERROR (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 101704 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 101704 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 101704 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 101704 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-28T01:07:22.641Z
[junit4] 2> 101775 INFO (zkConnectionManagerCallback-51-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 101818 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 101818 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-1-001/solr.xml
[junit4] 2> 101821 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 101821 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 101822 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 101877 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44406/solr
[junit4] 2> 101946 INFO (zkConnectionManagerCallback-55-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 101985 INFO (zkConnectionManagerCallback-57-thread-1-processing-n:127.0.0.1:45004_di%2Fri) [n:127.0.0.1:45004_di%2Fri ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 102028 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45004_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 102063 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45004_di%2Fri ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 102066 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45004_di%2Fri ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45004_di%2Fri
[junit4] 2> 102080 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 102093 INFO (zkCallback-38-thread-2-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 102218 INFO (zkCallback-56-thread-1-processing-n:127.0.0.1:45004_di%2Fri) [n:127.0.0.1:45004_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 102290 INFO (OverseerCollectionConfigSetProcessor-99424772207673349-127.0.0.1:36724_di%2Fri-n_0000000000) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 102875 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45004_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 103092 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45004_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 103092 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45004_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 103095 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45004_di%2Fri ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-1-001/cores
[junit4] 2> 103533 INFO (qtp524226505-248) [n:127.0.0.1:36724_di%2Fri ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:45004_di%252Fri&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 103749 INFO (OverseerThreadFactory-125-thread-3-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:45004_di%2Fri for creating new replica
[junit4] 2> 104887 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 104980 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 106231 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 106354 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema [collection1_shard1_replica_n21] Schema name=test
[junit4] 2> 106818 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 107204 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n21' using configuration from collection collection1, trusted=true
[junit4] 2> 107205 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n21' (registry 'solr.core.collection1.shard1.replica_n21') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 107205 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 107205 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [[collection1_shard1_replica_n21] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-1-001/cores/collection1_shard1_replica_n21], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-1-001/cores/collection1_shard1_replica_n21/data/]
[junit4] 2> 107221 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=37, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 107371 WARN (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 107815 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 107815 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 107817 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 107817 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 107874 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1727434677, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 107875 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.SolrIndexSearcher Opening [Searcher@573d36c5[collection1_shard1_replica_n21] main]
[junit4] 2> 107889 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 107909 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 107910 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 107910 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590796378541391872
[junit4] 2> 108052 INFO (searcherExecutor-139-thread-1-processing-n:127.0.0.1:45004_di%2Fri x:collection1_shard1_replica_n21 s:shard1 c:collection1) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SolrCore [collection1_shard1_replica_n21] Registered new searcher Searcher@573d36c5[collection1_shard1_replica_n21] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 108108 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 108109 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 108109 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:45004/di/ri/collection1_shard1_replica_n21/
[junit4] 2> 108109 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 108109 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.SyncStrategy https://127.0.0.1:45004/di/ri/collection1_shard1_replica_n21/ has no replicas
[junit4] 2> 108109 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 108181 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:45004/di/ri/collection1_shard1_replica_n21/ shard1
[junit4] 2> 108232 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 108254 INFO (qtp411175514-296) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n21] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n21&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3366
[junit4] 2> 108311 INFO (qtp524226505-248) [n:127.0.0.1:36724_di%2Fri ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:45004_di%252Fri&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=4778
[junit4] 2> 108987 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-2-001 of type NRT
[junit4] 2> 108989 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 108998 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 108998 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 108998 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 108999 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3a11a0e6{/di/ri,null,AVAILABLE}
[junit4] 2> 108999 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@69546e5{SSL,[ssl, http/1.1]}{127.0.0.1:45304}
[junit4] 2> 108999 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.Server Started @119502ms
[junit4] 2> 108999 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/tempDir-001/jetty2, solrconfig=solrconfig.xml, hostContext=/di/ri, hostPort=45304, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-2-001/cores}
[junit4] 2> 109000 ERROR (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 109034 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 109035 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 109035 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 109035 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-28T01:07:29.972Z
[junit4] 2> 109091 INFO (zkConnectionManagerCallback-61-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 109114 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 109114 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-2-001/solr.xml
[junit4] 2> 109148 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 109148 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 109169 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 109225 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44406/solr
[junit4] 2> 109318 INFO (zkConnectionManagerCallback-65-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 109383 INFO (zkConnectionManagerCallback-67-thread-1-processing-n:127.0.0.1:45304_di%2Fri) [n:127.0.0.1:45304_di%2Fri ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 109545 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45304_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 109566 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45304_di%2Fri ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 109584 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45304_di%2Fri ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45304_di%2Fri
[junit4] 2> 109585 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 109585 INFO (zkCallback-38-thread-2-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 109606 INFO (zkCallback-56-thread-1-processing-n:127.0.0.1:45004_di%2Fri) [n:127.0.0.1:45004_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 109691 INFO (zkCallback-66-thread-1-processing-n:127.0.0.1:45304_di%2Fri) [n:127.0.0.1:45304_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 109764 INFO (OverseerCollectionConfigSetProcessor-99424772207673349-127.0.0.1:36724_di%2Fri-n_0000000000) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 110095 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45304_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 110255 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45304_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 110256 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45304_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 110257 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:45304_di%2Fri ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-2-001/cores
[junit4] 2> 110616 INFO (qtp524226505-246) [n:127.0.0.1:36724_di%2Fri ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:45304_di%252Fri&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 110761 INFO (OverseerThreadFactory-125-thread-4-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:45304_di%2Fri for creating new replica
[junit4] 2> 110997 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 110997 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 112137 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 112197 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema [collection1_shard1_replica_n23] Schema name=test
[junit4] 2> 112902 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 113046 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n23' using configuration from collection collection1, trusted=true
[junit4] 2> 113047 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n23' (registry 'solr.core.collection1.shard1.replica_n23') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 113047 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 113047 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrCore [[collection1_shard1_replica_n23] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-2-001/cores/collection1_shard1_replica_n23], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-2-001/cores/collection1_shard1_replica_n23/data/]
[junit4] 2> 113083 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=37, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 113125 WARN (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 113632 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 113632 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=1000 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 113633 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 113633 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 113634 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1727434677, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 113635 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.SolrIndexSearcher Opening [Searcher@50109785[collection1_shard1_replica_n23] main]
[junit4] 2> 113648 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 113648 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 113649 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 113649 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590796384559169536
[junit4] 2> 113723 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.ZkController Core needs to recover:collection1_shard1_replica_n23
[junit4] 2> 113742 INFO (qtp455899797-327) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n23&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2744
[junit4] 2> 113843 INFO (searcherExecutor-150-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 x:collection1_shard1_replica_n23] o.a.s.c.SolrCore [collection1_shard1_replica_n23] Registered new searcher Searcher@50109785[collection1_shard1_replica_n23] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 113851 INFO (updateExecutor-62-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 113887 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 113960 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 113960 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard1_replica_n23]
[junit4] 2> 113960 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 113960 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard1_replica_n23] as recovering, leader is [https://127.0.0.1:45004/di/ri/collection1_shard1_replica_n21/] and I am [https://127.0.0.1:45304/di/ri/collection1_shard1_replica_n23/]
[junit4] 2> 114063 INFO (qtp524226505-246) [n:127.0.0.1:36724_di%2Fri ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:45304_di%252Fri&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=3446
[junit4] 2> 114098 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Sending prep recovery command to [https://127.0.0.1:45004/di/ri]; [WaitForState: action=PREPRECOVERY&core=collection1_shard1_replica_n21&nodeName=127.0.0.1:45304_di%252Fri&coreNodeName=core_node24&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 114965 INFO (OverseerCollectionConfigSetProcessor-99424772207673349-127.0.0.1:36724_di%2Fri-n_0000000000) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 115000 INFO (qtp411175514-297) [n:127.0.0.1:45004_di%2Fri ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node24, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
[junit4] 2> 115002 INFO (qtp411175514-297) [n:127.0.0.1:45004_di%2Fri ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard1, thisCore=collection1_shard1_replica_n21, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:45304_di%2Fri, coreNodeName=core_node24, onlyIfActiveCheckResult=false, nodeProps: core_node24:{"core":"collection1_shard1_replica_n23","base_url":"https://127.0.0.1:45304/di/ri","node_name":"127.0.0.1:45304_di%2Fri","state":"recovering","type":"NRT"}
[junit4] 2> 115002 INFO (qtp411175514-297) [n:127.0.0.1:45004_di%2Fri ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node24, state: recovering, checkLive: true, onlyIfLeader: true for: 0 seconds.
[junit4] 2> 115002 INFO (qtp411175514-297) [n:127.0.0.1:45004_di%2Fri ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:45304_di%252Fri&onlyIfLeaderActive=true&core=collection1_shard1_replica_n21&coreNodeName=core_node24&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 115564 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Attempting to PeerSync from [https://127.0.0.1:45004/di/ri/collection1_shard1_replica_n21/] - recoveringAfterStartup=[true]
[junit4] 2> 115617 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n23 url=https://127.0.0.1:45304/di/ri START replicas=[https://127.0.0.1:45004/di/ri/collection1_shard1_replica_n21/] nUpdates=1000
[junit4] 2> 115963 INFO (qtp411175514-294) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_n21] o.a.s.u.IndexFingerprint IndexFingerprint millis:124.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 115963 INFO (qtp411175514-294) [n:127.0.0.1:45004_di%2Fri c:collection1 s:shard1 r:core_node22 x:collection1_shard1_replica_n21] o.a.s.c.S.Request [collection1_shard1_replica_n21] webapp=/di/ri path=/get params={distrib=false&qt=/get&getFingerprint=9223372036854775807&wt=javabin&version=2} status=0 QTime=126
[junit4] 2> 116026 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.IndexFingerprint IndexFingerprint millis:61.0 result:{maxVersionSpecified=9223372036854775807, maxVersionEncountered=0, maxInHash=0, versionsHash=0, numVersions=0, numDocs=0, maxDoc=0}
[junit4] 2> 116026 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.PeerSync We are already in sync. No need to do a PeerSync
[junit4] 2> 116026 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 116026 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 116027 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 116027 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy PeerSync stage of recovery was successful.
[junit4] 2> 116027 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Replaying updates buffered during PeerSync.
[junit4] 2> 116027 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy No replay needed.
[junit4] 2> 116027 INFO (recoveryExecutor-63-thread-1-processing-n:127.0.0.1:45304_di%2Fri x:collection1_shard1_replica_n23 s:shard1 c:collection1 r:core_node24) [n:127.0.0.1:45304_di%2Fri c:collection1 s:shard1 r:core_node24 x:collection1_shard1_replica_n23] o.a.s.c.RecoveryStrategy Registering as Active after recovery.
[junit4] 2> 116470 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-3-001 of type NRT
[junit4] 2> 116543 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 116621 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 116621 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 116622 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 116642 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@48d1b059{/di/ri,null,AVAILABLE}
[junit4] 2> 116643 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@40f5abb5{SSL,[ssl, http/1.1]}{127.0.0.1:42602}
[junit4] 2> 116643 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.e.j.s.Server Started @127146ms
[junit4] 2> 116643 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/tempDir-001/jetty3, solrconfig=solrconfig.xml, hostContext=/di/ri, hostPort=42602, coreRootDirectory=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-3-001/cores}
[junit4] 2> 116643 ERROR (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 116694 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 116694 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 116694 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 116694 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-28T01:07:37.631Z
[junit4] 2> 116743 INFO (zkConnectionManagerCallback-72-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 116771 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 116771 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-3-001/solr.xml
[junit4] 2> 116774 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 116774 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 116793 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 116809 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44406/solr
[junit4] 2> 116943 INFO (zkConnectionManagerCallback-76-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 117034 INFO (zkConnectionManagerCallback-78-thread-1-processing-n:127.0.0.1:42602_di%2Fri) [n:127.0.0.1:42602_di%2Fri ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 117057 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:42602_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 117091 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:42602_di%2Fri ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 117094 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:42602_di%2Fri ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42602_di%2Fri
[junit4] 2> 117127 INFO (zkCallback-38-thread-1-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 117127 INFO (zkCallback-66-thread-1-processing-n:127.0.0.1:45304_di%2Fri) [n:127.0.0.1:45304_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 117127 INFO (zkCallback-56-thread-1-processing-n:127.0.0.1:45004_di%2Fri) [n:127.0.0.1:45004_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 117139 INFO (zkCallback-48-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 117234 INFO (zkCallback-77-thread-1-processing-n:127.0.0.1:42602_di%2Fri) [n:127.0.0.1:42602_di%2Fri ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 117642 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:42602_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 117754 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:42602_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 117755 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:42602_di%2Fri ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@21aa0f28
[junit4] 2> 117769 INFO (TEST-PeerSyncReplicationTest.test-seed#[4E1384E88A72F977]) [n:127.0.0.1:42602_di%2Fri ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/../../../../../../../../../../home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.PeerSyncReplicationTest_4E1384E88A72F977-001/shard-3-001/cores
[junit4] 2> 118154 INFO (qtp524226505-248) [n:127.0.0.1:36724_di%2Fri ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:42602_di%252Fri&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 118185 INFO (OverseerThreadFactory-125-thread-5-processing-n:127.0.0.1:36724_di%2Fri) [n:127.0.0.1:36724_di%2Fri ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:42602_di%2Fri for creating new replica
[junit4] 2> 118521 INFO (qtp1688903095-363) [n:127.0.0.1:42602_di%2Fri ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n25&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 118522 INFO (qtp1688903095-363) [n:127.0.0.
[...truncated too long message...]
ale.metrics.MetricRegistry@26dffd9e
[junit4] 2> 46021 T53 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 46021 T53 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@589ae5a9: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2aa9b9c1
[junit4] 2> 46022 T256 x:x:collection1 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7130db95
[junit4] 2> 46022 T256 x:x:collection1 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.core.collection1, tag=1899027349
[junit4] 2> 46022 T256 x:x:collection1 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1ed55fa4: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@25e1807
[junit4] 2> 46032 T53 oejs.AbstractConnector.doStop Stopped ServerConnector@22f5f964{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 46032 T53 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=124426710
[junit4] 2> 46032 T53 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 46032 T53 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@10af415a: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6e540848
[junit4] 2> 46037 T53 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 46037 T53 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7ca539e0: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@26dffd9e
[junit4] 2> 46040 T53 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 46040 T53 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1d30b9b6: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2aa9b9c1
[junit4] 2> 46041 T259 x:x:collection1 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@2b76f819
[junit4] 2> 46041 T259 x:x:collection1 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.core.collection1, tag=729217049
[junit4] 2> 46041 T259 x:x:collection1 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2ff8ffbc: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@6c3b68
[junit4] 2> 46051 T53 oejsh.ContextHandler.doStop Stopped o.e.j.s.ServletContextHandler@ced4608{/solr,null,UNAVAILABLE}
[junit4] 2> 46051 T53 oejss.HouseKeeper.stopScavenging Stopped scavenging
[junit4] 2> 46053 T53 oas.SolrTestCaseJ4.tearDown ###Ending testFullImportBadConfig
[junit4] 2> 46053 T53 oas.SolrTestCaseJ4.clearObjectTrackerAndCheckEmpty ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Jan 28, 2018 2:04:33 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Jan 28, 2018 2:04:53 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd:
[junit4] 2> 1) Thread[id=157, name=qtp1016972771-157, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Jan 28, 2018 2:04:53 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=157, name=qtp1016972771-157, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
[junit4] 2> Jan 28, 2018 2:04:56 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=157, name=qtp1016972771-157, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=HighCompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=28525, maxDocsPerChunk=4, blockSize=1), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=HIGH_COMPRESSION, chunkSize=28525, blockSize=1)), sim=RandomSimilarity(queryNorm=true): {}, locale=cs, timezone=AGT
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=3,free=159465816,total=322437120
[junit4] 2> NOTE: All tests run in this JVM: [TestZKPropertiesWriter, TestSolrEntityProcessorEndToEnd]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrEntityProcessorEndToEnd -Dtests.seed=B254D9F0868961D2 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=cs -Dtests.timezone=AGT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | TestSolrEntityProcessorEndToEnd (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd:
[junit4] > 1) Thread[id=157, name=qtp1016972771-157, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([B254D9F0868961D2]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=157, name=qtp1016972771-157, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([B254D9F0868961D2]:0)
[junit4] Completed [38/38 (1!)] on J0 in 61.73s, 7 tests, 2 errors <<< FAILURES!
[...truncated 42515 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 336 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/336/
9 tests failed.
FAILED: org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal
Error Message:
concurrent alias creation failed java.lang.IllegalStateException: Connection pool shut down
Stack Trace:
java.lang.AssertionError: concurrent alias creation failed java.lang.IllegalStateException: Connection pool shut down
at __randomizedtesting.SeedInfo.seed([5C443574D8348035:98DF3926752B370F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal(ConcurrentCreateRoutedAliasTest.java:108)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test
Error Message:
Expected 2 of 3 replicas to be active but only found 1; [core_node5:{"core":"c8n_1x3_lf_shard1_replica_t1","base_url":"http://127.0.0.1:36184/w/h","node_name":"127.0.0.1:36184_w%2Fh","state":"active","type":"TLOG","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/17)={ "pullReplicas":"0", "replicationFactor":"1", "shards":{"shard1":{ "range":"80000000-7fffffff", "state":"active", "replicas":{ "core_node4":{ "core":"c8n_1x3_lf_shard1_replica_t2", "base_url":"http://127.0.0.1:34913/w/h", "node_name":"127.0.0.1:34913_w%2Fh", "state":"down", "type":"TLOG"}, "core_node5":{ "core":"c8n_1x3_lf_shard1_replica_t1", "base_url":"http://127.0.0.1:36184/w/h", "node_name":"127.0.0.1:36184_w%2Fh", "state":"active", "type":"TLOG", "leader":"true"}, "core_node6":{ "state":"down", "base_url":"http://127.0.0.1:58802/w/h", "core":"c8n_1x3_lf_shard1_replica_t3", "node_name":"127.0.0.1:58802_w%2Fh", "type":"TLOG"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"0", "tlogReplicas":"3"}
Stack Trace:
java.lang.AssertionError: Expected 2 of 3 replicas to be active but only found 1; [core_node5:{"core":"c8n_1x3_lf_shard1_replica_t1","base_url":"http://127.0.0.1:36184/w/h","node_name":"127.0.0.1:36184_w%2Fh","state":"active","type":"TLOG","leader":"true"}]; clusterState: DocCollection(c8n_1x3_lf//collections/c8n_1x3_lf/state.json/17)={
"pullReplicas":"0",
"replicationFactor":"1",
"shards":{"shard1":{
"range":"80000000-7fffffff",
"state":"active",
"replicas":{
"core_node4":{
"core":"c8n_1x3_lf_shard1_replica_t2",
"base_url":"http://127.0.0.1:34913/w/h",
"node_name":"127.0.0.1:34913_w%2Fh",
"state":"down",
"type":"TLOG"},
"core_node5":{
"core":"c8n_1x3_lf_shard1_replica_t1",
"base_url":"http://127.0.0.1:36184/w/h",
"node_name":"127.0.0.1:36184_w%2Fh",
"state":"active",
"type":"TLOG",
"leader":"true"},
"core_node6":{
"state":"down",
"base_url":"http://127.0.0.1:58802/w/h",
"core":"c8n_1x3_lf_shard1_replica_t3",
"node_name":"127.0.0.1:58802_w%2Fh",
"type":"TLOG"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"0",
"tlogReplicas":"3"}
at __randomizedtesting.SeedInfo.seed([5C443574D8348035:D4100AAE76C8EDCD]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.testRf3WithLeaderFailover(LeaderFailoverAfterPartitionTest.java:169)
at org.apache.solr.cloud.LeaderFailoverAfterPartitionTest.test(LeaderFailoverAfterPartitionTest.java:56)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventFromRestoredState
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([5C443574D8348035:5C7281B4F139245F]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventFromRestoredState(TriggerIntegrationTest.java:760)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue.testDistributedQueue
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([5C443574D8348035]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.autoscaling.sim.TestGenericDistributedQueue
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([5C443574D8348035]:0)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([5C443574D8348035:10C2BFD17F2267A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue
Error Message:
action wasn't interrupted
Stack Trace:
java.lang.AssertionError: action wasn't interrupted
at __randomizedtesting.SeedInfo.seed([5C443574D8348035:95F177DAD15346C0]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventQueue(TestTriggerIntegration.java:640)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([5C443574D8348035:10C2BFD17F2267A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1178)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration
Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:10175_solr should have been deleted
Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:10175_solr should have been deleted
at __randomizedtesting.SeedInfo.seed([5C443574D8348035:44FEBD78D6014DDA]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration(TestTriggerIntegration.java:844)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11979 lines...]
[junit4] Suite: org.apache.solr.cloud.ConcurrentCreateRoutedAliasTest
[junit4] 2> 744650 INFO (SUITE-ConcurrentCreateRoutedAliasTest-seed#[5C443574D8348035]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_5C443574D8348035-001/init-core-data-001
[junit4] 2> 744651 WARN (SUITE-ConcurrentCreateRoutedAliasTest-seed#[5C443574D8348035]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 744651 INFO (SUITE-ConcurrentCreateRoutedAliasTest-seed#[5C443574D8348035]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 744661 INFO (SUITE-ConcurrentCreateRoutedAliasTest-seed#[5C443574D8348035]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 744677 INFO (TEST-ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal-seed#[5C443574D8348035]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testConcurrentCreateRoutedAliasMinimal
[junit4] 2> 744677 INFO (TEST-ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal-seed#[5C443574D8348035]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_5C443574D8348035-001/tempDir-001
[junit4] 2> 744678 INFO (TEST-ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal-seed#[5C443574D8348035]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 744713 INFO (Thread-245) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 744713 INFO (Thread-245) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 744813 INFO (TEST-ConcurrentCreateRoutedAliasTest.testConcurrentCreateRoutedAliasMinimal-seed#[5C443574D8348035]) [ ] o.a.s.c.ZkTestServer start zk server on port:34749
[junit4] 2> 744815 ERROR (Thread-245) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 744833 INFO (zkConnectionManagerCallback-299-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 744847 INFO (jetty-launcher-296-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 744868 INFO (jetty-launcher-296-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 744877 INFO (jetty-launcher-296-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 745163 INFO (jetty-launcher-296-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 745163 INFO (jetty-launcher-296-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 745163 INFO (jetty-launcher-296-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 745163 INFO (jetty-launcher-296-thread-3) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 745185 INFO (jetty-launcher-296-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 745185 INFO (jetty-launcher-296-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 745185 INFO (jetty-launcher-296-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 745226 INFO (jetty-launcher-296-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 745226 INFO (jetty-launcher-296-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 745226 INFO (jetty-launcher-296-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 745250 INFO (jetty-launcher-296-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@45bb31e3{/solr,null,AVAILABLE}
[junit4] 2> 745277 INFO (jetty-launcher-296-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@38f7dd56{SSL,[ssl, http/1.1]}{127.0.0.1:49517}
[junit4] 2> 745277 INFO (jetty-launcher-296-thread-2) [ ] o.e.j.s.Server Started @767236ms
[junit4] 2> 745277 INFO (jetty-launcher-296-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49517}
[junit4] 2> 745277 ERROR (jetty-launcher-296-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 745278 INFO (jetty-launcher-296-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 745278 INFO (jetty-launcher-296-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 745278 INFO (jetty-launcher-296-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 745278 INFO (jetty-launcher-296-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-27T15:39:03.476Z
[junit4] 2> 745377 INFO (jetty-launcher-296-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7571c670{/solr,null,AVAILABLE}
[junit4] 2> 745378 INFO (jetty-launcher-296-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@70da1530{SSL,[ssl, http/1.1]}{127.0.0.1:51692}
[junit4] 2> 745378 INFO (jetty-launcher-296-thread-1) [ ] o.e.j.s.Server Started @767337ms
[junit4] 2> 745378 INFO (jetty-launcher-296-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51692}
[junit4] 2> 745378 ERROR (jetty-launcher-296-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 745378 INFO (jetty-launcher-296-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 745378 INFO (jetty-launcher-296-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 745378 INFO (jetty-launcher-296-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 745379 INFO (jetty-launcher-296-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-27T15:39:03.577Z
[junit4] 2> 745581 INFO (jetty-launcher-296-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@70a2ed1f{/solr,null,AVAILABLE}
[junit4] 2> 745582 INFO (jetty-launcher-296-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4fc2e6f{SSL,[ssl, http/1.1]}{127.0.0.1:57844}
[junit4] 2> 745582 INFO (jetty-launcher-296-thread-3) [ ] o.e.j.s.Server Started @767541ms
[junit4] 2> 745582 INFO (jetty-launcher-296-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57844}
[junit4] 2> 745582 ERROR (jetty-launcher-296-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 745582 INFO (jetty-launcher-296-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 745582 INFO (jetty-launcher-296-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 745582 INFO (jetty-launcher-296-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 745583 INFO (jetty-launcher-296-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-27T15:39:03.780Z
[junit4] 2> 745661 INFO (jetty-launcher-296-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 745661 INFO (jetty-launcher-296-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 745661 INFO (jetty-launcher-296-thread-4) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 745693 INFO (jetty-launcher-296-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5efa29f3{/solr,null,AVAILABLE}
[junit4] 2> 745694 INFO (jetty-launcher-296-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@189f556d{SSL,[ssl, http/1.1]}{127.0.0.1:49747}
[junit4] 2> 745694 INFO (jetty-launcher-296-thread-4) [ ] o.e.j.s.Server Started @767653ms
[junit4] 2> 745694 INFO (jetty-launcher-296-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=49747}
[junit4] 2> 745694 ERROR (jetty-launcher-296-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 745694 INFO (jetty-launcher-296-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 745694 INFO (jetty-launcher-296-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 745694 INFO (jetty-launcher-296-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 745694 INFO (jetty-launcher-296-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-27T15:39:03.892Z
[junit4] 2> 745707 INFO (zkConnectionManagerCallback-303-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 745765 INFO (jetty-launcher-296-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 745813 INFO (zkConnectionManagerCallback-305-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 745814 INFO (jetty-launcher-296-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34749/solr
[junit4] 2> 745815 INFO (zkConnectionManagerCallback-301-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 745827 INFO (jetty-launcher-296-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 745838 INFO (jetty-launcher-296-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 745855 INFO (jetty-launcher-296-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34749/solr
[junit4] 2> 745857 INFO (zkConnectionManagerCallback-307-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 745857 INFO (zkConnectionManagerCallback-311-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 745866 INFO (jetty-launcher-296-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 745893 INFO (jetty-launcher-296-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34749/solr
[junit4] 2> 745894 INFO (zkConnectionManagerCallback-315-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 745907 INFO (jetty-launcher-296-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34749/solr
[junit4] 2> 745934 INFO (zkConnectionManagerCallback-317-thread-1-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 745966 INFO (zkConnectionManagerCallback-321-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 746129 INFO (zkConnectionManagerCallback-325-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 746225 INFO (zkConnectionManagerCallback-327-thread-1-processing-n:127.0.0.1:49517_solr) [n:127.0.0.1:49517_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 746226 INFO (zkConnectionManagerCallback-330-thread-1-processing-n:127.0.0.1:49747_solr) [n:127.0.0.1:49747_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 746226 INFO (zkConnectionManagerCallback-332-thread-1-processing-n:127.0.0.1:57844_solr) [n:127.0.0.1:57844_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 746345 INFO (jetty-launcher-296-thread-1) [n:127.0.0.1:51692_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 746346 INFO (jetty-launcher-296-thread-1) [n:127.0.0.1:51692_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51692_solr
[junit4] 2> 746346 INFO (jetty-launcher-296-thread-1) [n:127.0.0.1:51692_solr ] o.a.s.c.Overseer Overseer (id=99422538493001735-127.0.0.1:51692_solr-n_0000000000) starting
[junit4] 2> 746349 INFO (jetty-launcher-296-thread-2) [n:127.0.0.1:49517_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 746373 INFO (jetty-launcher-296-thread-3) [n:127.0.0.1:57844_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 746453 INFO (jetty-launcher-296-thread-2) [n:127.0.0.1:49517_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49517_solr
[junit4] 2> 746473 INFO (jetty-launcher-296-thread-3) [n:127.0.0.1:57844_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57844_solr
[junit4] 2> 746698 INFO (jetty-launcher-296-thread-4) [n:127.0.0.1:49747_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 746708 INFO (jetty-launcher-296-thread-4) [n:127.0.0.1:49747_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 746742 INFO (jetty-launcher-296-thread-4) [n:127.0.0.1:49747_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:49747_solr
[junit4] 2> 747702 INFO (jetty-launcher-296-thread-1) [n:127.0.0.1:51692_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51692_solr
[junit4] 2> 748258 INFO (zkCallback-329-thread-1-processing-n:127.0.0.1:49747_solr) [n:127.0.0.1:49747_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
[junit4] 2> 748258 INFO (zkCallback-331-thread-1-processing-n:127.0.0.1:57844_solr) [n:127.0.0.1:57844_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 748258 INFO (zkCallback-326-thread-1-processing-n:127.0.0.1:49517_solr) [n:127.0.0.1:49517_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 748298 INFO (OverseerStateUpdate-99422538493001735-127.0.0.1:51692_solr-n_0000000000) [n:127.0.0.1:51692_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 748624 INFO (jetty-launcher-296-thread-2) [n:127.0.0.1:49517_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49517.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 748803 INFO (jetty-launcher-296-thread-1) [n:127.0.0.1:51692_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51692.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 748810 INFO (jetty-launcher-296-thread-1) [n:127.0.0.1:51692_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51692.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 748810 INFO (jetty-launcher-296-thread-1) [n:127.0.0.1:51692_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51692.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 748813 INFO (jetty-launcher-296-thread-2) [n:127.0.0.1:49517_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49517.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 748822 INFO (jetty-launcher-296-thread-2) [n:127.0.0.1:49517_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49517.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 748825 INFO (jetty-launcher-296-thread-2) [n:127.0.0.1:49517_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_5C443574D8348035-001/tempDir-001/node2/.
[junit4] 2> 748826 INFO (jetty-launcher-296-thread-1) [n:127.0.0.1:51692_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_5C443574D8348035-001/tempDir-001/node1/.
[junit4] 2> 762306 WARN (jetty-launcher-296-thread-2-SendThread(127.0.0.1:34749)) [n:127.0.0.1:49517_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 13393ms for sessionid 0x1613845a18b000a
[junit4] 2> 762317 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1613845a18b000a, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 762306 WARN (jetty-launcher-296-thread-3-SendThread(127.0.0.1:34749)) [n:127.0.0.1:57844_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 14380ms for sessionid 0x1613845a18b000b
[junit4] 2> 762306 WARN (jetty-launcher-296-thread-4-SendThread(127.0.0.1:34749)) [n:127.0.0.1:49747_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 14525ms for sessionid 0x1613845a18b000c
[junit4] 2> 762824 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1613845a18b000b, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 762824 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1613845a18b000c, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 762407 WARN (zkConnectionManagerCallback-327-thread-1-processing-n:127.0.0.1:49517_solr) [n:127.0.0.1:49517_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@55385a6f name: ZooKeeperConnection Watcher:127.0.0.1:34749/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 762850 WARN (zkConnectionManagerCallback-327-thread-1-processing-n:127.0.0.1:49517_solr) [n:127.0.0.1:49517_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 762977 WARN (zkConnectionManagerCallback-332-thread-1-processing-n:127.0.0.1:57844_solr) [n:127.0.0.1:57844_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@7cd76c41 name: ZooKeeperConnection Watcher:127.0.0.1:34749/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 762977 WARN (zkConnectionManagerCallback-332-thread-1-processing-n:127.0.0.1:57844_solr) [n:127.0.0.1:57844_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 763188 INFO (jetty-launcher-296-thread-4) [n:127.0.0.1:49747_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49747.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 763221 WARN (zkConnectionManagerCallback-330-thread-1-processing-n:127.0.0.1:49747_solr) [n:127.0.0.1:49747_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@782aeb3e name: ZooKeeperConnection Watcher:127.0.0.1:34749/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 763221 WARN (zkConnectionManagerCallback-330-thread-1-processing-n:127.0.0.1:49747_solr) [n:127.0.0.1:49747_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 763271 INFO (jetty-launcher-296-thread-4) [n:127.0.0.1:49747_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49747.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 763271 INFO (jetty-launcher-296-thread-4) [n:127.0.0.1:49747_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_49747.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 763333 INFO (jetty-launcher-296-thread-4) [n:127.0.0.1:49747_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_5C443574D8348035-001/tempDir-001/node4/.
[junit4] 2> 763627 INFO (jetty-launcher-296-thread-3) [n:127.0.0.1:57844_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57844.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 763681 INFO (zkConnectionManagerCallback-327-thread-1-processing-n:127.0.0.1:49517_solr) [n:127.0.0.1:49517_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 763700 INFO (jetty-launcher-296-thread-3) [n:127.0.0.1:57844_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57844.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 763700 INFO (jetty-launcher-296-thread-3) [n:127.0.0.1:57844_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57844.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@773fdfc5
[junit4] 2> 763711 INFO (jetty-launcher-296-thread-3) [n:127.0.0.1:57844_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.ConcurrentCreateRoutedAliasTest_5C443574D8348035-001/tempDir-001/node3/.
[junit4] 2> 764106 INFO (zkConnectionManagerCallback-330-thread-1-processing-n:127.0.0.1:49747_solr) [n:127.0.0.1:49747_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 764701 INFO (zkConnectionManagerCallback-332-thread-1-processing-n:127.0.0.1:57844_solr) [n:127.0.0.1:57844_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 765145 INFO (zkConnectionManagerCallback-341-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 768245 INFO (qtp194051025-1303) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias5&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 768764 INFO (qtp194051025-1532) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias32&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 769297 INFO (qtp194051025-1302) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias27&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 769365 INFO (qtp194051025-1516) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias38&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 769367 INFO (qtp194051025-1526) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias1&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 769391 INFO (OverseerThreadFactory-607-thread-1-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias5_2018-01-27_15
[junit4] 2> 769405 INFO (OverseerThreadFactory-607-thread-5-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias38_2018-01-27_15
[junit4] 2> 769406 INFO (OverseerThreadFactory-607-thread-4-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias1_2018-01-27_15
[junit4] 2> 769407 INFO (OverseerThreadFactory-607-thread-2-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias32_2018-01-27_15
[junit4] 2> 769407 INFO (OverseerThreadFactory-607-thread-3-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias27_2018-01-27_15
[junit4] 2> 769447 INFO (qtp194051025-1545) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias47&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 769789 INFO (qtp194051025-1547) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias4&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 769791 INFO (OverseerThreadFactory-607-thread-6-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias47_2018-01-27_15
[junit4] 2> 769798 INFO (qtp194051025-1549) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias42&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 769798 INFO (qtp194051025-1530) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias8&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 769799 INFO (qtp194051025-1517) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias24&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 769497 INFO (OverseerStateUpdate-99422538493001735-127.0.0.1:51692_solr-n_0000000000) [n:127.0.0.1:51692_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testAlias5_2018-01-27_15",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testAlias5_2018-01-27_15_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49517/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 769894 INFO (qtp194051025-1541) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias20&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 769989 INFO (qtp194051025-1555) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias30&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770134 INFO (qtp194051025-1550) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias12&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770134 INFO (qtp194051025-1527) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias49&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770135 INFO (qtp194051025-1299) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias13&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770135 INFO (qtp194051025-1528) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias31&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770478 INFO (OverseerThreadFactory-607-thread-7-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias4_2018-01-27_15
[junit4] 2> 770522 INFO (qtp194051025-1519) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias48&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770523 INFO (qtp194051025-1531) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias6&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770529 INFO (qtp194051025-1551) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias18&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770636 INFO (OverseerThreadFactory-607-thread-8-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias24_2018-01-27_15
[junit4] 2> 770640 INFO (qtp194051025-1556) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias35&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770957 INFO (qtp194051025-1518) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias10&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770962 INFO (qtp194051025-1525) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias9&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770962 INFO (qtp194051025-1538) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias7&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 770962 INFO (qtp194051025-1297) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias37&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771336 INFO (qtp194051025-1569) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias29&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771347 INFO (OverseerStateUpdate-99422538493001735-127.0.0.1:51692_solr-n_0000000000) [n:127.0.0.1:51692_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testAlias27_2018-01-27_15",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testAlias27_2018-01-27_15_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:51692/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 771348 INFO (OverseerStateUpdate-99422538493001735-127.0.0.1:51692_solr-n_0000000000) [n:127.0.0.1:51692_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testAlias1_2018-01-27_15",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testAlias1_2018-01-27_15_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49747/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 771349 INFO (OverseerStateUpdate-99422538493001735-127.0.0.1:51692_solr-n_0000000000) [n:127.0.0.1:51692_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testAlias38_2018-01-27_15",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testAlias38_2018-01-27_15_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49747/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 771349 INFO (OverseerStateUpdate-99422538493001735-127.0.0.1:51692_solr-n_0000000000) [n:127.0.0.1:51692_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testAlias32_2018-01-27_15",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testAlias32_2018-01-27_15_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:57844/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 771351 INFO (qtp194051025-1563) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias23&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771360 INFO (OverseerThreadFactory-607-thread-9-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias42_2018-01-27_15
[junit4] 2> 771366 INFO (qtp194051025-1554) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias14&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771366 INFO (qtp194051025-1553) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias0&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771366 INFO (qtp194051025-1521) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias19&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771377 INFO (qtp194051025-1537) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias2&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771377 INFO (qtp194051025-1529) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias44&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771378 INFO (qtp194051025-1559) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias41&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771378 INFO (qtp194051025-1300) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias34&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771378 INFO (qtp194051025-1301) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias45&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771379 INFO (qtp194051025-1536) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias40&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771379 INFO (qtp194051025-1296) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias16&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771380 INFO (OverseerThreadFactory-607-thread-10-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias8_2018-01-27_15
[junit4] 2> 771380 INFO (qtp194051025-1522) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias15&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771380 INFO (qtp194051025-1562) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias25&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771380 INFO (qtp194051025-1535) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias22&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771381 INFO (qtp194051025-1534) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias43&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771381 INFO (qtp194051025-1561) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias17&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771393 INFO (qtp194051025-1564) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias3&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771466 INFO (qtp194051025-1520) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias26&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771489 INFO (qtp194051025-1565) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias46&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771490 INFO (qtp194051025-1566) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias21&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771490 INFO (qtp194051025-1567) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias11&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771491 INFO (qtp194051025-1568) [n:127.0.0.1:51692_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createroutedalias with params router.interval=%2B12HOUR&router.start=NOW/HOUR&create-collection.collection.configName=_default&router.field=routedFoo_dt&name=testAlias39&router.name=time&create-collection.numShards=1&create-collection.nrtReplicas=1&action=CREATEROUTEDALIAS&wt=javabin&version=2&create-collection.replicationFactor=1 and sendToOCPQueue=true
[junit4] 2> 771685 INFO (OverseerStateUpdate-99422538493001735-127.0.0.1:51692_solr-n_0000000000) [n:127.0.0.1:51692_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testAlias42_2018-01-27_15",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testAlias42_2018-01-27_15_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49517/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 771686 INFO (OverseerStateUpdate-99422538493001735-127.0.0.1:51692_solr-n_0000000000) [n:127.0.0.1:51692_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testAlias24_2018-01-27_15",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testAlias24_2018-01-27_15_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49747/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 771687 INFO (OverseerStateUpdate-99422538493001735-127.0.0.1:51692_solr-n_0000000000) [n:127.0.0.1:51692_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testAlias8_2018-01-27_15",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testAlias8_2018-01-27_15_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49747/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 771687 INFO (OverseerStateUpdate-99422538493001735-127.0.0.1:51692_solr-n_0000000000) [n:127.0.0.1:51692_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testAlias47_2018-01-27_15",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testAlias47_2018-01-27_15_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:49747/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 771699 INFO (OverseerThreadFactory-607-thread-11-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias20_2018-01-27_15
[junit4] 2> 771743 INFO (OverseerThreadFactory-607-thread-13-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias12_2018-01-27_15
[junit4] 2> 771745 INFO (OverseerThreadFactory-607-thread-12-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias30_2018-01-27_15
[junit4] 2> 771761 INFO (OverseerThreadFactory-607-thread-14-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias13_2018-01-27_15
[junit4] 2> 771784 INFO (OverseerThreadFactory-607-thread-15-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias31_2018-01-27_15
[junit4] 2> 771807 INFO (OverseerThreadFactory-607-thread-16-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias49_2018-01-27_15
[junit4] 2> 771816 INFO (OverseerThreadFactory-607-thread-17-processing-n:127.0.0.1:51692_solr) [n:127.0.0.1:51692_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testAlias48_2018-01-27_15
[junit4] 2> 771847 INFO (OverseerThreadFactory-607-thread-18-processing-n:127.0.0.1:51692_solr
[...truncated too long message...]
ve/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/bcprov-jdk15on-1.54.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/commons-math3-3.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/analysis/icu/lucene-analyzers-icu-7.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/contrib/analysis-extras/lib/icu4j-60.2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180127_152615_9113758609875533046473.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180127_152615_9114747300054153322530.suites -stdin
[junit4] ERROR: JVM J1 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180127_152615_9113654611955155589331.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180127_152615_911628447201174378206.syserr
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
[junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] at java.lang.Thread.run(Thread.java:748)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:836: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:780: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build.xml:262: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/common-build.xml:556: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:1513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:1038: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J0-20180127_152615_9036502585241868026825.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J0-20180127_152615_9035344044204316037534.syserr
Total time: 501 minutes 11 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-7.x #335 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-7.x - Build # 335 - Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/335/
5 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestCloudJSONFacetJoinDomain
Error Message:
Could not load collection from ZK: org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection
at __randomizedtesting.SeedInfo.seed([39B3207A8764B2DF]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:137)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
at org.apache.solr.cloud.TestCloudJSONFacetJoinDomain.waitForRecoveriesToFinish(TestCloudJSONFacetJoinDomain.java:806)
at org.apache.solr.cloud.TestCloudJSONFacetJoinDomain.createMiniSolrCloudCluster(TestCloudJSONFacetJoinDomain.java:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:874)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
... 29 more
FAILED: org.apache.solr.cloud.TestUtilizeNode.test
Error Message:
no replica should be present in 127.0.0.1:44718_solr
Stack Trace:
java.lang.AssertionError: no replica should be present in 127.0.0.1:44718_solr
at __randomizedtesting.SeedInfo.seed([39B3207A8764B2DF:B1E71FA02998DF27]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.TestUtilizeNode.test(TestUtilizeNode.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testCooldown
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([39B3207A8764B2DF:80D4D9EF9CEC72D]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testCooldown(TriggerIntegrationTest.java:1177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([39B3207A8764B2DF:544F84873D2C4DD8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:241)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([39B3207A8764B2DF:83BF17F5D88C6490]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.junit.Assert.assertNull(Assert.java:562)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12129 lines...]
[junit4] Suite: org.apache.solr.cloud.TestCloudJSONFacetJoinDomain
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestCloudJSONFacetJoinDomain_39B3207A8764B2DF-001/init-core-data-001
[junit4] 2> 1015031 WARN (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 1015031 INFO (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1015032 INFO (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1015033 INFO (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1015046 INFO (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestCloudJSONFacetJoinDomain_39B3207A8764B2DF-001/tempDir-001
[junit4] 2> 1015046 INFO (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1015047 INFO (Thread-1358) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1015047 INFO (Thread-1358) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1015087 ERROR (Thread-1358) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1015151 INFO (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:45533
[junit4] 2> 1015179 INFO (zkConnectionManagerCallback-911-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1015235 INFO (jetty-launcher-908-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T17:27:37-04:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1015235 INFO (jetty-launcher-908-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1015235 INFO (jetty-launcher-908-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1015235 INFO (jetty-launcher-908-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1015235 INFO (jetty-launcher-908-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@537896fc{/solr,null,AVAILABLE}
[junit4] 2> 1015305 INFO (jetty-launcher-908-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@413bd421{SSL,[ssl, http/1.1]}{127.0.0.1:37564}
[junit4] 2> 1015305 INFO (jetty-launcher-908-thread-1) [ ] o.e.j.s.Server Started @1025711ms
[junit4] 2> 1015305 INFO (jetty-launcher-908-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37564}
[junit4] 2> 1015306 ERROR (jetty-launcher-908-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1015306 INFO (jetty-launcher-908-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 1015306 INFO (jetty-launcher-908-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1015306 INFO (jetty-launcher-908-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1015306 INFO (jetty-launcher-908-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-27T00:09:06.467Z
[junit4] 2> 1015338 INFO (zkConnectionManagerCallback-913-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1015351 INFO (jetty-launcher-908-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1015395 INFO (jetty-launcher-908-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45533/solr
[junit4] 2> 1015398 INFO (zkConnectionManagerCallback-917-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1015409 INFO (zkConnectionManagerCallback-919-thread-1-processing-n:127.0.0.1:37564_solr) [n:127.0.0.1:37564_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1015877 INFO (jetty-launcher-908-thread-1) [n:127.0.0.1:37564_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1015878 INFO (jetty-launcher-908-thread-1) [n:127.0.0.1:37564_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37564_solr
[junit4] 2> 1015949 INFO (jetty-launcher-908-thread-1) [n:127.0.0.1:37564_solr ] o.a.s.c.Overseer Overseer (id=99418881802043395-127.0.0.1:37564_solr-n_0000000000) starting
[junit4] 2> 1016140 INFO (jetty-launcher-908-thread-1) [n:127.0.0.1:37564_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37564_solr
[junit4] 2> 1016143 INFO (zkCallback-918-thread-1-processing-n:127.0.0.1:37564_solr) [n:127.0.0.1:37564_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1016510 INFO (jetty-launcher-908-thread-1) [n:127.0.0.1:37564_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37564.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@160ce943
[junit4] 2> 1016545 INFO (jetty-launcher-908-thread-1) [n:127.0.0.1:37564_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37564.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@160ce943
[junit4] 2> 1016545 INFO (jetty-launcher-908-thread-1) [n:127.0.0.1:37564_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37564.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@160ce943
[junit4] 2> 1016546 INFO (jetty-launcher-908-thread-1) [n:127.0.0.1:37564_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestCloudJSONFacetJoinDomain_39B3207A8764B2DF-001/tempDir-001/node1/.
[junit4] 2> 1016587 INFO (zkConnectionManagerCallback-923-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1016602 INFO (zkConnectionManagerCallback-927-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1016603 INFO (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1016603 INFO (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45533/solr ready
[junit4] 2> 1017701 INFO (qtp1409660235-5526) [n:127.0.0.1:37564_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&property.config=solrconfig-tlog.xml&property.schema=schema_latest.xml&collection.configName=org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_config-set&name=org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1017720 INFO (OverseerThreadFactory-2745-thread-1-processing-n:127.0.0.1:37564_solr) [n:127.0.0.1:37564_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection
[junit4] 2> 1017892 INFO (OverseerStateUpdate-99418881802043395-127.0.0.1:37564_solr-n_0000000000) [n:127.0.0.1:37564_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37564/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1018181 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_config-set&newCollection=true&collection=org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema_latest.xml&coreNodeName=core_node2&name=org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin
[junit4] 2> 1018181 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1018558 INFO (zkCallback-918-thread-1-processing-n:127.0.0.1:37564_solr) [n:127.0.0.1:37564_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection/state.json] for collection [org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1018559 INFO (zkCallback-918-thread-2-processing-n:127.0.0.1:37564_solr) [n:127.0.0.1:37564_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection/state.json] for collection [org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1019512 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.3.0
[junit4] 2> 1019622 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.s.IndexSchema [org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] Schema name=example
[junit4] 2> 1020019 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema example/1.6 with uniqueid field id
[junit4] 2> 1020089 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1' using configuration from collection org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection, trusted=true
[junit4] 2> 1020103 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37564.solr.core.org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection.shard1.replica_n1' (registry 'solr.core.org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@160ce943
[junit4] 2> 1020103 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1020103 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.SolrCore [[org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestCloudJSONFacetJoinDomain_39B3207A8764B2DF-001/tempDir-001/node1/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.TestCloudJSONFacetJoinDomain_39B3207A8764B2DF-001/tempDir-001/node1/./org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1/data/]
[junit4] 2> 1020106 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=24, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 1020140 WARN (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1020427 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1020427 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1020428 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1020428 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1020446 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=11, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.700460739531595]
[junit4] 2> 1020447 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4941609d[org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] main]
[junit4] 2> 1020519 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_config-set
[junit4] 2> 1020556 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_config-set
[junit4] 2> 1020556 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1020557 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1590702114569453568
[junit4] 2> 1020559 INFO (searcherExecutor-2748-thread-1-processing-n:127.0.0.1:37564_solr x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1 s:shard1 c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection r:core_node2) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.SolrCore [org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] Registered new searcher Searcher@4941609d[org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1020610 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1020610 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1020610 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:37564/solr/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1/
[junit4] 2> 1020610 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1020610 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:37564/solr/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 1020610 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1020612 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:37564/solr/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1/ shard1
[junit4] 2> 1020718 INFO (zkCallback-918-thread-2-processing-n:127.0.0.1:37564_solr) [n:127.0.0.1:37564_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection/state.json] for collection [org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1020718 INFO (zkCallback-918-thread-1-processing-n:127.0.0.1:37564_solr) [n:127.0.0.1:37564_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection/state.json] for collection [org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1020750 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1020803 INFO (qtp1409660235-5522) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_config-set&newCollection=true&collection=org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection&version=2&replicaType=NRT&property.config=solrconfig-tlog.xml&property.schema=schema_latest.xml&coreNodeName=core_node2&name=org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&wt=javabin} status=0 QTime=2622
[junit4] 2> 1027985 WARN (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker-SendThread(127.0.0.1:45533)) [ ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 10206ms for sessionid 0x16134f23dc70005
[junit4] 2> 1028048 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x16134f23dc70005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1028095 INFO (zkCallback-918-thread-2-processing-n:127.0.0.1:37564_solr) [n:127.0.0.1:37564_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection/state.json] for collection [org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1028095 INFO (zkCallback-918-thread-1-processing-n:127.0.0.1:37564_solr) [n:127.0.0.1:37564_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection/state.json] for collection [org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1028099 INFO (qtp1409660235-5526) [n:127.0.0.1:37564_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1028167 WARN (zkConnectionManagerCallback-927-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@13fb4590 name: ZooKeeperConnection Watcher:127.0.0.1:45533/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 1028167 WARN (zkConnectionManagerCallback-927-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 1029107 INFO (qtp1409660235-5526) [n:127.0.0.1:37564_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&property.config=solrconfig-tlog.xml&property.schema=schema_latest.xml&collection.configName=org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_config-set&name=org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=11405
[junit4] 2> 1029107 INFO (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection failOnTimeout:true timeout (sec):330
[junit4] 1> -
[junit4] 2> 1029415 WARN (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker-SendThread(127.0.0.1:45533)) [ ] o.a.z.ClientCnxn Unable to reconnect to ZooKeeper service, session 0x16134f23dc70005 has expired
[junit4] 2> 1029427 WARN (zkConnectionManagerCallback-927-thread-1) [ ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@13fb4590 name: ZooKeeperConnection Watcher:127.0.0.1:45533/solr got event WatchedEvent state:Expired type:None path:null path: null type: None
[junit4] 2> 1029427 WARN (zkConnectionManagerCallback-927-thread-1) [ ] o.a.s.c.c.ConnectionManager Our previous ZooKeeper session was expired. Attempting to reconnect to recover relationship with ZooKeeper...
[junit4] 2> 1029427 WARN (zkConnectionManagerCallback-927-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Connection expired - starting a new one...
[junit4] 2> 1029460 INFO (zkConnectionManagerCallback-927-thread-1-EventThread) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1029460 INFO (zkConnectionManagerCallback-927-thread-1) [ ] o.a.s.c.c.ConnectionManager Connection with ZooKeeper reestablished.
[junit4] 2> 1029462 INFO (zkConnectionManagerCallback-927-thread-1) [ ] o.a.s.c.c.DefaultConnectionStrategy Reconnected to ZooKeeper
[junit4] 2> 1029462 INFO (zkConnectionManagerCallback-927-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient Connected:true
[junit4] 2> 1029569 INFO (jetty-closer-909-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@413bd421{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1029569 INFO (jetty-closer-909-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1673904333
[junit4] 2> 1029621 INFO (jetty-closer-909-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1029622 INFO (jetty-closer-909-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@298db0f: rootName = solr_37564, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@77d1b85c
[junit4] 2> 1029637 INFO (jetty-closer-909-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1029638 INFO (jetty-closer-909-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@a26f85b: rootName = solr_37564, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1c579c46
[junit4] 2> 1029640 INFO (jetty-closer-909-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1029640 INFO (jetty-closer-909-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4998a6ed: rootName = solr_37564, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@18680b14
[junit4] 2> 1029642 INFO (jetty-closer-909-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37564_solr
[junit4] 2> 1029643 INFO (jetty-closer-909-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1029643 INFO (zkCallback-918-thread-2-processing-n:127.0.0.1:37564_solr) [n:127.0.0.1:37564_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1029666 INFO (coreCloseExecutor-2753-thread-1) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.c.SolrCore [org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@ce14d2f
[junit4] 2> 1029666 INFO (coreCloseExecutor-2753-thread-1) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection.shard1.replica_n1, tag=216091951
[junit4] 2> 1029666 INFO (coreCloseExecutor-2753-thread-1) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@69fb10a7: rootName = solr_37564, domain = solr.core.org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@11aa05a0
[junit4] 2> 1029672 INFO (coreCloseExecutor-2753-thread-1) [n:127.0.0.1:37564_solr c:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection s:shard1 r:core_node2 x:org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection.shard1.leader, tag=216091951
[junit4] 2> 1029695 INFO (jetty-closer-909-thread-1) [ ] o.a.s.c.Overseer Overseer (id=99418881802043395-127.0.0.1:37564_solr-n_0000000000) closing
[junit4] 2> 1029695 WARN (OverseerAutoScalingTriggerThread-99418881802043395-127.0.0.1:37564_solr-n_0000000000) [n:127.0.0.1:37564_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1029698 INFO (OverseerStateUpdate-99418881802043395-127.0.0.1:37564_solr-n_0000000000) [n:127.0.0.1:37564_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37564_solr
[junit4] 2> 1029759 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x16134f23dc70003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1029763 INFO (jetty-closer-909-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@537896fc{/solr,null,UNAVAILABLE}
[junit4] 2> 1029763 INFO (jetty-closer-909-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1029833 ERROR (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1029848 INFO (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45533 45533
[junit4] 2> 1029954 INFO (Thread-1358) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45533 45533
[junit4] 2> 1029955 WARN (Thread-1358) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/collections/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection/state.json
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 1029955 INFO (SUITE-TestCloudJSONFacetJoinDomain-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1523, maxMBSortInHeap=6.7920742367185865, sim=RandomSimilarity(queryNorm=false): {}, locale=zh-HK, timezone=America/Matamoros
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=270459600,total=524288000
[junit4] 2> NOTE: All tests run in this JVM: [SystemInfoHandlerTest, DistributedExpandComponentTest, TestPushWriter, TestReplicationHandler, TestExactStatsCache, QueryElevationComponentTest, TestSweetSpotSimilarityFactory, LeaderElectionContextKeyTest, TestCollapseQParserPlugin, HighlighterTest, TestRecovery, ConfigSetsAPITest, TestDynamicFieldResource, TestManagedSchemaThreadSafety, TlogReplayBufferedWhileIndexingTest, DirectUpdateHandlerOptimizeTest, BigEndianAscendingWordSerializerTest, AlternateDirectoryTest, MigrateRouteKeyTest, OverseerCollectionConfigSetProcessorTest, ShardRoutingCustomTest, TestDefaultStatsCache, TestBulkSchemaAPI, TestRestoreCore, TestSimpleQParserPlugin, DeleteStatusTest, ShardSplitTest, TestLeaderElectionZkExpiry, TestRetrieveFieldsOptimizer, SortByFunctionTest, SecurityConfHandlerTest, SolrSlf4jReporterTest, CdcrVersionReplicationTest, TestSearchPerf, TestCloudJSONFacetJoinDomain]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCloudJSONFacetJoinDomain -Dtests.seed=39B3207A8764B2DF -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=zh-HK -Dtests.timezone=America/Matamoros -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestCloudJSONFacetJoinDomain (suite) <<<
[junit4] > Throwable #1: org.apache.solr.common.SolrException: Could not load collection from ZK: org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection
[junit4] > at __randomizedtesting.SeedInfo.seed([39B3207A8764B2DF]:0)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
[junit4] > at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:137)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:154)
[junit4] > at org.apache.solr.cloud.TestCloudJSONFacetJoinDomain.waitForRecoveriesToFinish(TestCloudJSONFacetJoinDomain.java:806)
[junit4] > at org.apache.solr.cloud.TestCloudJSONFacetJoinDomain.createMiniSolrCloudCluster(TestCloudJSONFacetJoinDomain.java:119)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/org.apache.solr.cloud.TestCloudJSONFacetJoinDomain_collection/state.json
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] > at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
[junit4] > at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] > at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
[junit4] > at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
[junit4] > ... 29 more
[junit4] Completed [137/769 (1!)] on J1 in 15.00s, 0 tests, 1 error <<< FAILURES!
[...truncated 1350 lines...]
[junit4] Suite: org.apache.solr.cloud.TestUtilizeNode
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_39B3207A8764B2DF-001/init-core-data-001
[junit4] 2> 3349365 WARN (SUITE-TestUtilizeNode-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1213 numCloses=1213
[junit4] 2> 3349365 INFO (SUITE-TestUtilizeNode-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 3349366 INFO (SUITE-TestUtilizeNode-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 3349366 INFO (SUITE-TestUtilizeNode-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 3349379 INFO (SUITE-TestUtilizeNode-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_39B3207A8764B2DF-001/tempDir-001
[junit4] 2> 3349379 INFO (SUITE-TestUtilizeNode-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3349387 INFO (Thread-4198) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3349387 INFO (Thread-4198) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3349428 ERROR (Thread-4198) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3349493 INFO (SUITE-TestUtilizeNode-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:42184
[junit4] 2> 3349500 INFO (zkConnectionManagerCallback-6584-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3349570 INFO (jetty-launcher-6581-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3349572 INFO (jetty-launcher-6581-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3349573 INFO (jetty-launcher-6581-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3349590 INFO (jetty-launcher-6581-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 3349591 INFO (jetty-launcher-6581-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3349591 INFO (jetty-launcher-6581-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3349591 INFO (jetty-launcher-6581-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 3349591 INFO (jetty-launcher-6581-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@64773885{/solr,null,AVAILABLE}
[junit4] 2> 3349592 INFO (jetty-launcher-6581-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@13f202bf{SSL,[ssl, http/1.1]}{127.0.0.1:44470}
[junit4] 2> 3349592 INFO (jetty-launcher-6581-thread-1) [ ] o.e.j.s.Server Started @3360070ms
[junit4] 2> 3349592 INFO (jetty-launcher-6581-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44470}
[junit4] 2> 3349592 ERROR (jetty-launcher-6581-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3349592 INFO (jetty-launcher-6581-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 3349592 INFO (jetty-launcher-6581-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3349592 INFO (jetty-launcher-6581-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3349592 INFO (jetty-launcher-6581-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-27T00:48:00.835Z
[junit4] 2> 3349606 INFO (jetty-launcher-6581-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3349606 INFO (jetty-launcher-6581-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3349606 INFO (jetty-launcher-6581-thread-3) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 3349607 INFO (jetty-launcher-6581-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3349607 INFO (jetty-launcher-6581-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3349607 INFO (jetty-launcher-6581-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 3349608 INFO (zkConnectionManagerCallback-6586-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3349609 INFO (jetty-launcher-6581-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3349672 INFO (jetty-launcher-6581-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1457943d{/solr,null,AVAILABLE}
[junit4] 2> 3349672 INFO (jetty-launcher-6581-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@be5e9f5{/solr,null,AVAILABLE}
[junit4] 2> 3349675 INFO (jetty-launcher-6581-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 3349675 INFO (jetty-launcher-6581-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 3349675 INFO (jetty-launcher-6581-thread-4) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 3349675 INFO (jetty-launcher-6581-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@422fb1a{/solr,null,AVAILABLE}
[junit4] 2> 3349676 INFO (jetty-launcher-6581-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@52d31b38{SSL,[ssl, http/1.1]}{127.0.0.1:37787}
[junit4] 2> 3349676 INFO (jetty-launcher-6581-thread-4) [ ] o.e.j.s.Server Started @3360154ms
[junit4] 2> 3349676 INFO (jetty-launcher-6581-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37787}
[junit4] 2> 3349676 ERROR (jetty-launcher-6581-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3349676 INFO (jetty-launcher-6581-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 3349676 INFO (jetty-launcher-6581-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3349676 INFO (jetty-launcher-6581-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3349676 INFO (jetty-launcher-6581-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-27T00:48:00.919Z
[junit4] 2> 3349727 INFO (zkConnectionManagerCallback-6588-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3349727 INFO (jetty-launcher-6581-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3349852 INFO (jetty-launcher-6581-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@60b5bd2a{SSL,[ssl, http/1.1]}{127.0.0.1:38417}
[junit4] 2> 3349852 INFO (jetty-launcher-6581-thread-3) [ ] o.e.j.s.Server Started @3360330ms
[junit4] 2> 3349852 INFO (jetty-launcher-6581-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38417}
[junit4] 2> 3349852 ERROR (jetty-launcher-6581-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3349852 INFO (jetty-launcher-6581-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 3349852 INFO (jetty-launcher-6581-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3349852 INFO (jetty-launcher-6581-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3349852 INFO (jetty-launcher-6581-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-27T00:48:01.095Z
[junit4] 2> 3349865 INFO (jetty-launcher-6581-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@563c9802{SSL,[ssl, http/1.1]}{127.0.0.1:40574}
[junit4] 2> 3349867 INFO (jetty-launcher-6581-thread-2) [ ] o.e.j.s.Server Started @3360345ms
[junit4] 2> 3349867 INFO (jetty-launcher-6581-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40574}
[junit4] 2> 3349867 ERROR (jetty-launcher-6581-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3349882 INFO (jetty-launcher-6581-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42184/solr
[junit4] 2> 3349893 INFO (jetty-launcher-6581-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.3.0
[junit4] 2> 3350019 INFO (jetty-launcher-6581-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3350019 INFO (jetty-launcher-6581-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3350019 INFO (jetty-launcher-6581-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-27T00:48:01.262Z
[junit4] 2> 3350019 INFO (jetty-launcher-6581-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42184/solr
[junit4] 2> 3349965 INFO (zkConnectionManagerCallback-6594-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3350021 INFO (jetty-launcher-6581-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3350022 INFO (zkConnectionManagerCallback-6600-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3350047 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1613515dc760005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3350104 INFO (zkConnectionManagerCallback-6592-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3350128 INFO (jetty-launcher-6581-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42184/solr
[junit4] 2> 3350141 INFO (zkConnectionManagerCallback-6598-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3350230 INFO (zkConnectionManagerCallback-6602-thread-1-processing-n:127.0.0.1:37787_solr) [n:127.0.0.1:37787_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3350230 INFO (zkConnectionManagerCallback-6608-thread-1-processing-n:127.0.0.1:44470_solr) [n:127.0.0.1:44470_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3350286 INFO (zkConnectionManagerCallback-6606-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3350287 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x1613515dc760008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3350287 INFO (jetty-launcher-6581-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3350326 INFO (zkConnectionManagerCallback-6611-thread-1-processing-n:127.0.0.1:38417_solr) [n:127.0.0.1:38417_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3350424 INFO (jetty-launcher-6581-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42184/solr
[junit4] 2> 3350432 INFO (jetty-launcher-6581-thread-4) [n:127.0.0.1:37787_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3350433 INFO (jetty-launcher-6581-thread-4) [n:127.0.0.1:37787_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37787_solr
[junit4] 2> 3350434 INFO (jetty-launcher-6581-thread-4) [n:127.0.0.1:37787_solr ] o.a.s.c.Overseer Overseer (id=99419034788167687-127.0.0.1:37787_solr-n_0000000000) starting
[junit4] 2> 3350448 INFO (zkConnectionManagerCallback-6616-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3350525 INFO (zkConnectionManagerCallback-6618-thread-1-processing-n:127.0.0.1:40574_solr) [n:127.0.0.1:40574_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3350528 INFO (jetty-launcher-6581-thread-4) [n:127.0.0.1:37787_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37787_solr
[junit4] 2> 3350590 INFO (OverseerStateUpdate-99419034788167687-127.0.0.1:37787_solr-n_0000000000) [n:127.0.0.1:37787_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3350627 INFO (jetty-launcher-6581-thread-3) [n:127.0.0.1:38417_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3350629 INFO (jetty-launcher-6581-thread-3) [n:127.0.0.1:38417_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3350672 INFO (jetty-launcher-6581-thread-2) [n:127.0.0.1:40574_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 3350674 INFO (jetty-launcher-6581-thread-2) [n:127.0.0.1:40574_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3350708 INFO (jetty-launcher-6581-thread-2) [n:127.0.0.1:40574_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40574_solr
[junit4] 2> 3350712 INFO (jetty-launcher-6581-thread-3) [n:127.0.0.1:38417_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38417_solr
[junit4] 2> 3350837 INFO (zkCallback-6601-thread-2-processing-n:127.0.0.1:37787_solr) [n:127.0.0.1:37787_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3350838 INFO (zkCallback-6601-thread-2-processing-n:127.0.0.1:37787_solr) [n:127.0.0.1:37787_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3350838 INFO (zkCallback-6617-thread-2-processing-n:127.0.0.1:40574_solr) [n:127.0.0.1:40574_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 3350951 INFO (zkCallback-6610-thread-1-processing-n:127.0.0.1:38417_solr) [n:127.0.0.1:38417_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 3351056 INFO (zkCallback-6610-thread-2-processing-n:127.0.0.1:38417_solr) [n:127.0.0.1:38417_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3351197 INFO (jetty-launcher-6581-thread-1) [n:127.0.0.1:44470_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3351203 INFO (jetty-launcher-6581-thread-3) [n:127.0.0.1:38417_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38417.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351220 INFO (jetty-launcher-6581-thread-2) [n:127.0.0.1:40574_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40574.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351260 INFO (jetty-launcher-6581-thread-1) [n:127.0.0.1:44470_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3351276 INFO (jetty-launcher-6581-thread-1) [n:127.0.0.1:44470_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44470_solr
[junit4] 2> 3351277 INFO (jetty-launcher-6581-thread-4) [n:127.0.0.1:37787_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37787.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351301 INFO (zkCallback-6601-thread-2-processing-n:127.0.0.1:37787_solr) [n:127.0.0.1:37787_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3351301 INFO (zkCallback-6610-thread-2-processing-n:127.0.0.1:38417_solr) [n:127.0.0.1:38417_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3351352 INFO (zkCallback-6607-thread-1-processing-n:127.0.0.1:44470_solr) [n:127.0.0.1:44470_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3351352 INFO (zkCallback-6617-thread-1-processing-n:127.0.0.1:40574_solr) [n:127.0.0.1:40574_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3351357 INFO (jetty-launcher-6581-thread-2) [n:127.0.0.1:40574_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40574.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351357 INFO (jetty-launcher-6581-thread-2) [n:127.0.0.1:40574_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40574.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351359 INFO (jetty-launcher-6581-thread-2) [n:127.0.0.1:40574_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_39B3207A8764B2DF-001/tempDir-001/node2/.
[junit4] 2> 3351373 INFO (jetty-launcher-6581-thread-3) [n:127.0.0.1:38417_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38417.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351373 INFO (jetty-launcher-6581-thread-3) [n:127.0.0.1:38417_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38417.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351374 INFO (jetty-launcher-6581-thread-3) [n:127.0.0.1:38417_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_39B3207A8764B2DF-001/tempDir-001/node3/.
[junit4] 2> 3351409 INFO (jetty-launcher-6581-thread-4) [n:127.0.0.1:37787_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37787.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351409 INFO (jetty-launcher-6581-thread-4) [n:127.0.0.1:37787_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37787.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351410 INFO (jetty-launcher-6581-thread-4) [n:127.0.0.1:37787_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_39B3207A8764B2DF-001/tempDir-001/node4/.
[junit4] 2> 3351519 INFO (jetty-launcher-6581-thread-1) [n:127.0.0.1:44470_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44470.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351594 INFO (jetty-launcher-6581-thread-1) [n:127.0.0.1:44470_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44470.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351594 INFO (jetty-launcher-6581-thread-1) [n:127.0.0.1:44470_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44470.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4932c3e3
[junit4] 2> 3351628 INFO (jetty-launcher-6581-thread-1) [n:127.0.0.1:44470_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_39B3207A8764B2DF-001/tempDir-001/node1/.
[junit4] 2> 3352308 INFO (zkConnectionManagerCallback-6626-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3352344 INFO (zkConnectionManagerCallback-6630-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3352344 INFO (SUITE-TestUtilizeNode-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 3352345 INFO (SUITE-TestUtilizeNode-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42184/solr ready
[junit4] 2> 3352537 INFO (qtp1719663402-23473) [n:127.0.0.1:40574_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3352717 INFO (qtp1719663402-23473) [n:127.0.0.1:40574_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=180
[junit4] 2> 3352938 INFO (TEST-TestUtilizeNode.test-seed#[39B3207A8764B2DF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 3352938 INFO (TEST-TestUtilizeNode.test-seed#[39B3207A8764B2DF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 3352972 INFO (zkConnectionManagerCallback-6633-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3352975 INFO (qtp1719663402-23472) [n:127.0.0.1:40574_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&name=utilizenodecoll&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3353073 INFO (OverseerCollectionConfigSetProcessor-99419034788167687-127.0.0.1:37787_solr-n_0000000000) [n:127.0.0.1:37787_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3353074 INFO (OverseerThreadFactory-7655-thread-2-processing-n:127.0.0.1:37787_solr) [n:127.0.0.1:37787_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection utilizenodecoll
[junit4] 2> 3353181 INFO (OverseerStateUpdate-99419034788167687-127.0.0.1:37787_solr-n_0000000000) [n:127.0.0.1:37787_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"utilizenodecoll_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:44470/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3353211 INFO (OverseerStateUpdate-99419034788167687-127.0.0.1:37787_solr-n_0000000000) [n:127.0.0.1:37787_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"utilizenodecoll_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37787/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3353241 INFO (OverseerStateUpdate-99419034788167687-127.0.0.1:37787_solr-n_0000000000) [n:127.0.0.1:37787_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"utilizenodecoll_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38417/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3353243 INFO (OverseerStateUpdate-99419034788167687-127.0.0.1:37787_solr-n_0000000000) [n:127.0.0.1:37787_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"utilizenodecoll_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40574/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3353504 INFO (qtp1719663402-23467) [n:127.0.0.1:40574_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard2_replica_n6&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3353505 INFO (qtp1719663402-23467) [n:127.0.0.1:40574_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3353774 INFO (qtp2140752914-23452) [n:127.0.0.1:44470_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard1_replica_n1&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3353792 INFO (qtp1821075250-23483) [n:127.0.0.1:37787_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard1_replica_n2&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3353792 INFO (qtp1821075250-23483) [n:127.0.0.1:37787_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3353791 INFO (qtp695893262-23464) [n:127.0.0.1:38417_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard2_replica_n4&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3353792 INFO (qtp695893262-23464) [n:127.0.0.1:38417_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3353803 INFO (qtp2140752914-23452) [n:127.0.0.1:44470_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3353966 INFO (zkCallback-6610-thread-2-processing-n:127.0.0.1:38417_solr) [n:127.0.0.1:38417_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3353966 INFO (zkCallback-6617-thread-1-processing-n:127.0.0.1:40574_solr) [n:127.0.0.1:40574_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 3353966 INFO (zkCallback-6617-thread-2-processing-n:127.0.0.1:40574_solr) [n:127.0.0
[...truncated too long message...]
FO (coreCloseExecutor-8452-thread-2) [n:127.0.0.1:39359_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testMetricTrigger.shard2.leader, tag=1155370841
[junit4] 2> 3850460 INFO (coreCloseExecutor-8454-thread-1) [n:127.0.0.1:38697_solr c:collection1 s:shard1 r:core_node12 x:collection1_shard1_replica_n11] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=672402598
[junit4] 2> 3850461 INFO (jetty-closer-7066-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@66e43397{/solr,null,UNAVAILABLE}
[junit4] 2> 3850461 INFO (coreCloseExecutor-8454-thread-2) [n:127.0.0.1:38697_solr c:collection1 s:shard1 r:core_node14 x:collection1_shard1_replica_n13] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1195924979
[junit4] 2> 3850463 INFO (zkCallback-7171-thread-3-processing-n:127.0.0.1:38697_solr) [n:127.0.0.1:38697_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38697_solr
[junit4] 2> 3850463 INFO (jetty-closer-7066-thread-4) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3850465 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x161351a77170020, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3850465 INFO (jetty-closer-7066-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3eb9c545{/solr,null,UNAVAILABLE}
[junit4] 2> 3850467 INFO (jetty-closer-7066-thread-5) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 3850467 ERROR (SUITE-TriggerIntegrationTest-seed#[39B3207A8764B2DF]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3850468 INFO (SUITE-TriggerIntegrationTest-seed#[39B3207A8764B2DF]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40209 40209
[junit4] 2> 3850548 INFO (Thread-4595) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40209 40209
[junit4] 2> 3850553 WARN (Thread-4595) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 19 /solr/clusterprops.json
[junit4] 2> 19 /solr/aliases.json
[junit4] 2> 18 /solr/security.json
[junit4] 2> 8 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 155 /solr/collections/collection1/state.json
[junit4] 2> 19 /solr/clusterstate.json
[junit4] 2> 11 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 6 /solr/overseer_elect/election/99419054562279456-127.0.0.1:38697_solr-n_0000000009
[junit4] 2> 3 /solr/collections/collection1/leader_elect/shard1/election/99419054562279450-core_node4-n_0000000000
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/99419054562279453-core_node8-n_0000000003
[junit4] 2> 2 /solr/overseer_elect/election/99419054562279447-127.0.0.1:45525_solr-n_0000000006
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 19 /solr/live_nodes
[junit4] 2> 19 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/queue
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_39B3207A8764B2DF-001
[junit4] 2> Jan 27, 2018 12:56:21 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=es-PE, timezone=Canada/Pacific
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=129478752,total=524288000
[junit4] 2> NOTE: All tests run in this JVM: [OverseerRolesTest, CachingDirectoryFactoryTest, DeleteShardTest, TestSort, TestDocBasedVersionConstraints, DistributedDebugComponentTest, TestNumericRangeQuery64, HttpPartitionTest, PeerSyncReplicationTest, SpatialHeatmapFacetsTest, ChaosMonkeyNothingIsSafeTest, OutputWriterTest, FullSolrCloudDistribCmdsTest, TestOrdValues, DistributedFacetPivotLargeTest, CollectionReloadTest, VersionInfoTest, TestExtendedDismaxParser, TestEmbeddedSolrServerSchemaAPI, XmlUpdateRequestHandlerTest, SolrGangliaReporterTest, JvmMetricsTest, SolrPluginUtilsTest, TestLegacyField, HLLUtilTest, TestSolrQueryParser, TestClusterProperties, EnumFieldTest, TestFreeTextSuggestions, DistributedTermsComponentTest, TestLockTree, DeleteLastCustomShardedReplicaTest, TestWordDelimiterFilterFactory, SpellCheckCollatorTest, TestSQLHandlerNonCloud, OverriddenZkACLAndCredentialsProvidersTest, TestNumericTerms32, TestUseDocValuesAsStored2, TestConfigSetImmutable, VMParamsZkACLAndCredentialsProvidersTest, TestQueryWrapperFilter, TestCloudPseudoReturnFields, TestAuthorizationFramework, TestRawTransformer, ResponseHeaderTest, TestCSVLoader, DistributedQueryElevationComponentTest, TestLeaderElectionWithEmptyReplica, TestTriggerIntegration, TimeRoutedAliasUpdateProcessorTest, DistributedFacetPivotSmallTest, TestPivotHelperCode, SharedFSAutoReplicaFailoverTest, MBeansHandlerTest, AssignBackwardCompatibilityTest, UninvertDocValuesMergePolicyTest, TestTestInjection, UpdateParamsTest, TestStandardQParsers, TestReRankQParserPlugin, IndexBasedSpellCheckerTest, CoreMergeIndexesAdminHandlerTest, TestGroupingSearch, ClassificationUpdateProcessorTest, TestChildDocTransformer, HighlighterConfigTest, SmileWriterTest, TestExactSharedStatsCache, QueryResultKeyTest, CollectionsAPIDistributedZkTest, SchemaApiFailureTest, NodeLostTriggerTest, MultiThreadedOCPTest, DocumentAnalysisRequestHandlerTest, ChaosMonkeySafeLeaderWithPullReplicasTest, TestMacroExpander, IgnoreCommitOptimizeUpdateProcessorFactoryTest, SolrShardReporterTest, TestSolr4Spatial2, TestSolrCloudSnapshots, TestCSVResponseWriter, TestManagedStopFilterFactory, SolrMetricManagerTest, TestIBSimilarityFactory, TestJavabinTupleStreamParser, TestRestManager, BinaryUpdateRequestHandlerTest, DirectUpdateHandlerTest, StressHdfsTest, CdcrBootstrapTest, SolrRequestParserTest, TestLegacyTerms, TestPseudoReturnFields, CloudExitableDirectoryReaderTest, SuggesterWFSTTest, CreateCollectionCleanupTest, LeaderFailureAfterFreshStartTest, TestDistribDocBasedVersion, BasicDistributedZkTest, TestScoreJoinQPNoScore, TestSearcherReuse, TestManagedSchemaAPI, HttpTriggerListenerTest, TestFieldResource, TestFastOutputStream, TestBinaryField, TestOmitPositions, TestComponentsName, SolrIndexConfigTest, TestExportWriter, HdfsNNFailoverTest, TestRequestForwarding, ZkStateWriterTest, TestRandomFaceting, CursorPagingTest, DistributedVersionInfoTest, AtomicUpdatesTest, TestDocumentBuilder, TestInitQParser, StatsReloadRaceTest, TestHashPartitioner, RulesTest, ChaosMonkeySafeLeaderTest, BasicDistributedZk2Test, UnloadDistributedZkTest, SyncSliceTest, ShardRoutingTest, ZkCLITest, TestDistributedGrouping, ZkControllerTest, TestStressReorder, TestJoin, TestStressVersions, TestSolr4Spatial, SolrCmdDistributorTest, TestFiltering, SoftAutoCommitTest, TestMergePolicyConfig, TestSolrDeletionPolicy2, MultiTermTest, TestDocSet, MinimalSchemaTest, TestElisionMultitermQuery, TestConfig, TestFuzzyAnalyzedSuggestions, ExternalFileFieldSortTest, TestSolrCoreProperties, TestPostingsSolrHighlighter, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestLuceneMatchVersion, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestCharFilters, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestDFRSimilarityFactory, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestLMJelinekMercerSimilarityFactory, TestBM25SimilarityFactory, TimeZoneUtilsTest, ResourceLoaderTest, OpenExchangeRatesOrgProviderTest, PluginInfoTest, TestFastLRUCache, PreAnalyzedFieldTest, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, ZkNodePropsTest, UUIDFieldTest, TestRTGBase, DistributedIntervalFacetingTest, TestTolerantSearch, TestEmbeddedSolrServerAdminHandler, ConnectionReuseTest, ActionThrottleTest, AddReplicaTest, CleanupOldIndexTest, CreateRoutedAliasTest, DeleteReplicaTest, DocValuesNotIndexedTest, LegacyCloudClusterPropTest, MoveReplicaTest, SaslZkACLProviderTest, SolrCLIZkUtilsTest, SolrCloudExampleTest, TestConfigSetsAPI, TestPullReplica, TestStressLiveNodes, TestTolerantUpdateProcessorCloud, TestTolerantUpdateProcessorRandomCloud, TestUtilizeNode, TriLevelCompositeIdRoutingTest, AssignTest, HdfsCollectionsAPIDistributedZkTest, SimpleCollectionCreateDeleteTest, TestCollectionsAPIViaSolrCloudCluster, AutoAddReplicasIntegrationTest, TriggerIntegrationTest]
[junit4] Completed [679/769 (4!)] on J0 in 199.84s, 14 tests, 2 failures <<< FAILURES!
[...truncated 48439 lines...]
[JENKINS] Lucene-Solr-Tests-7.x - Build # 334 - Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/334/
11 tests failed.
FAILED: org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
We have shards using the same indexDir. E.g. shards [core.test.shard2, core.test.shard1, core..system] all use indexDir (closed)
Stack Trace:
java.lang.AssertionError: We have shards using the same indexDir. E.g. shards [core.test.shard2, core.test.shard1, core..system] all use indexDir (closed)
at __randomizedtesting.SeedInfo.seed([A95B73A61123EBCC:E12E07121710C459]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.checkNoTwoShardsUseTheSameIndexDir(CollectionsAPIDistributedZkTest.java:604)
at org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:479)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple
Error Message:
Could not load collection from ZK: testSimple1
Stack Trace:
org.apache.solr.common.SolrException: Could not load collection from ZK: testSimple1
at __randomizedtesting.SeedInfo.seed([A95B73A61123EBCC:91E8575836D03F1D]:0)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1108)
at org.apache.solr.common.cloud.ZkStateReader$LazyCollectionRef.get(ZkStateReader.java:647)
at org.apache.solr.common.cloud.ClusterState.getCollectionOrNull(ClusterState.java:137)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:117)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.getReplacedSharedFsReplicas(AutoAddReplicasIntegrationTest.java:169)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:86)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /collections/testSimple1/state.json
at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1212)
at org.apache.solr.common.cloud.SolrZkClient.lambda$getData$5(SolrZkClient.java:339)
at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:339)
at org.apache.solr.common.cloud.ZkStateReader.fetchCollectionState(ZkStateReader.java:1120)
at org.apache.solr.common.cloud.ZkStateReader.getCollectionLive(ZkStateReader.java:1106)
... 44 more
FAILED: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:40224_solr, 127.0.0.1:54026_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/15)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node3":{ "dataDir":"hdfs://localhost:49060/data/testSimple1/core_node3/data/", "base_url":"http://127.0.0.1:54026/solr", "node_name":"127.0.0.1:54026_solr", "type":"NRT", "ulogDir":"hdfs://localhost:49060/data/testSimple1/core_node3/data/tlog", "core":"testSimple1_shard1_replica_n1", "shared_storage":"true", "state":"active"}, "core_node5":{ "dataDir":"hdfs://localhost:49060/data/testSimple1/core_node5/data/", "base_url":"http://127.0.0.1:54026/solr", "node_name":"127.0.0.1:54026_solr", "type":"NRT", "ulogDir":"hdfs://localhost:49060/data/testSimple1/core_node5/data/tlog", "core":"testSimple1_shard1_replica_n2", "shared_storage":"true", "state":"active", "leader":"true"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node7":{ "dataDir":"hdfs://localhost:49060/data/testSimple1/core_node7/data/", "base_url":"http://127.0.0.1:59191/solr", "node_name":"127.0.0.1:59191_solr", "type":"NRT", "ulogDir":"hdfs://localhost:49060/data/testSimple1/core_node7/data/tlog", "core":"testSimple1_shard2_replica_n4", "shared_storage":"true", "state":"down"}, "core_node8":{ "dataDir":"hdfs://localhost:49060/data/testSimple1/core_node8/data/", "base_url":"http://127.0.0.1:54026/solr", "node_name":"127.0.0.1:54026_solr", "type":"NRT", "ulogDir":"hdfs://localhost:49060/data/testSimple1/core_node8/data/tlog", "core":"testSimple1_shard2_replica_n6", "shared_storage":"true", "state":"active", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"true", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:40224_solr, 127.0.0.1:54026_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/15)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node3":{
"dataDir":"hdfs://localhost:49060/data/testSimple1/core_node3/data/",
"base_url":"http://127.0.0.1:54026/solr",
"node_name":"127.0.0.1:54026_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost:49060/data/testSimple1/core_node3/data/tlog",
"core":"testSimple1_shard1_replica_n1",
"shared_storage":"true",
"state":"active"},
"core_node5":{
"dataDir":"hdfs://localhost:49060/data/testSimple1/core_node5/data/",
"base_url":"http://127.0.0.1:54026/solr",
"node_name":"127.0.0.1:54026_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost:49060/data/testSimple1/core_node5/data/tlog",
"core":"testSimple1_shard1_replica_n2",
"shared_storage":"true",
"state":"active",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node7":{
"dataDir":"hdfs://localhost:49060/data/testSimple1/core_node7/data/",
"base_url":"http://127.0.0.1:59191/solr",
"node_name":"127.0.0.1:59191_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost:49060/data/testSimple1/core_node7/data/tlog",
"core":"testSimple1_shard2_replica_n4",
"shared_storage":"true",
"state":"down"},
"core_node8":{
"dataDir":"hdfs://localhost:49060/data/testSimple1/core_node8/data/",
"base_url":"http://127.0.0.1:54026/solr",
"node_name":"127.0.0.1:54026_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost:49060/data/testSimple1/core_node8/data/tlog",
"core":"testSimple1_shard2_replica_n6",
"shared_storage":"true",
"state":"active",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"true",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([A95B73A61123EBCC:91E8575836D03F1D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTriggerRestoreState
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([A95B73A61123EBCC:2166FAD92BE30A61]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeAddedTriggerRestoreState(TriggerIntegrationTest.java:426)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([A95B73A61123EBCC:135744294ECB3D83]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.junit.Assert.assertNull(Assert.java:562)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([A95B73A61123EBCC:F4136D2FDEE54D83]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1178)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testTriggerThrottling
Error Message:
Two TriggerAction instances should have been created by now
Stack Trace:
java.lang.AssertionError: Two TriggerAction instances should have been created by now
at __randomizedtesting.SeedInfo.seed([A95B73A61123EBCC:5279DB83C389085E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testTriggerThrottling(TestTriggerIntegration.java:167)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventFromRestoredState
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([A95B73A61123EBCC:A96DC766382E4FA6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventFromRestoredState(TestTriggerIntegration.java:690)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testCooldown
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([A95B73A61123EBCC:98E51E426F899E3E]:0)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testCooldown(TestTriggerIntegration.java:1086)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest
Error Message:
Clean up static fields (in @AfterClass?), your test seems to hang on to approximately 14,941,400 bytes (threshold is 10,485,760). Field reference sizes (counted individually): - 14,940,784 bytes, private static javax.management.MBeanServer org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer - 192 bytes, public static org.junit.rules.TestRule org.apache.solr.SolrTestCaseJ4.solrClassRules - 128 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.factoryProp - 112 bytes, private static java.lang.String org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION - 72 bytes, private static java.util.Map org.apache.solr.SolrTestCaseJ4.savedClassLogLevels - 64 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.coreName - 48 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.initialRootLogLevel
Stack Trace:
junit.framework.AssertionFailedError: Clean up static fields (in @AfterClass?), your test seems to hang on to approximately 14,941,400 bytes (threshold is 10,485,760). Field reference sizes (counted individually):
- 14,940,784 bytes, private static javax.management.MBeanServer org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.mBeanServer
- 192 bytes, public static org.junit.rules.TestRule org.apache.solr.SolrTestCaseJ4.solrClassRules
- 128 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.factoryProp
- 112 bytes, private static java.lang.String org.apache.solr.metrics.reporters.SolrJmxReporterCloudTest.COLLECTION
- 72 bytes, private static java.util.Map org.apache.solr.SolrTestCaseJ4.savedClassLogLevels
- 64 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.coreName
- 48 bytes, private static java.lang.String org.apache.solr.SolrTestCaseJ4.initialRootLogLevel
at __randomizedtesting.SeedInfo.seed([A95B73A61123EBCC]:0)
at com.carrotsearch.randomizedtesting.rules.StaticFieldsInvariantRule$1.afterAlways(StaticFieldsInvariantRule.java:170)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([A95B73A61123EBCC:F4136D2FDEE54D83]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11879 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_A95B73A61123EBCC-001/init-core-data-001
[junit4] 2> 760792 INFO (SUITE-TestTriggerIntegration-seed#[A95B73A61123EBCC]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 760849 INFO (SUITE-TestTriggerIntegration-seed#[A95B73A61123EBCC]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 760849 INFO (SUITE-TestTriggerIntegration-seed#[A95B73A61123EBCC]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 761108 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 761130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 761130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 761130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 761130 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 761139 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
[junit4] 2> 761141 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 761241 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 761241 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 761241 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 761242 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:154)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 761256 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 761256 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 761256 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 761256 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 761256 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 761263 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 761263 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 761267 DEBUG (ScheduledTrigger-1327-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 761307 DEBUG (ScheduledTrigger-1327-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 761757 DEBUG (ScheduledTrigger-1327-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 761783 DEBUG (ScheduledTrigger-1327-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 761823 DEBUG (ScheduledTrigger-1327-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 761843 DEBUG (ScheduledTrigger-1327-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 761847 DEBUG (simCloudManagerPool-1326-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 762005 DEBUG (ScheduledTrigger-1327-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 762006 DEBUG (simCloudManagerPool-1326-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 762006 DEBUG (simCloudManagerPool-1326-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 762008 INFO (simCloudManagerPool-1326-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 762008 DEBUG (simCloudManagerPool-1326-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 762008 DEBUG (simCloudManagerPool-1326-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 762008 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 762009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 762009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 762011 DEBUG (ScheduledTrigger-1327-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 762011 DEBUG (ScheduledTrigger-1327-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 762025 DEBUG (ScheduledTrigger-1327-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 762047 DEBUG (ScheduledTrigger-1327-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 762047 DEBUG (ScheduledTrigger-1327-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 762047 DEBUG (ScheduledTrigger-1327-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 762048 DEBUG (ScheduledTrigger-1327-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10002_solr at time 17220861845900398
[junit4] 2> 762087 DEBUG (ScheduledTrigger-1327-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 762087 DEBUG (ScheduledTrigger-1327-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 762087 DEBUG (ScheduledTrigger-1327-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 762087 DEBUG (ScheduledTrigger-1327-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10002_solr] added at times [17220861845900398], now=17220863833738648
[junit4] 2> 762088 DEBUG (ScheduledTrigger-1327-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"3d2e48590d0c6eTbthhb7oewwdldqsdeqsexzo9u",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":17220861845900398,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[17220861845900398],
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 762088 DEBUG (ScheduledTrigger-1327-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 762185 DEBUG (AutoscalingActionExecutor-1328-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"3d2e48590d0c6eTbthhb7oewwdldqsdeqsexzo9u",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":17220861845900398,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[17220861845900398],
[junit4] 2> "_enqueue_time_":17220863899672698,
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 762186 INFO (AutoscalingActionExecutor-1328-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"3d2e48590d0c6eTbthhb7oewwdldqsdeqsexzo9u",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":17220861845900398,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[17220861845900398],
[junit4] 2> "_enqueue_time_":17220863899672698,
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 762191 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 762191 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 762192 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 762192 WARN (AutoscalingActionExecutor-1328-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger1 has been closed.
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.dequeue(ScheduledTriggers.java:526)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:310)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 762607 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 762607 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
[junit4] 2> 762607 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 762607 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
[junit4] 2> 762608 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 762608 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion -1
[junit4] 2> 762608 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 762608 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10001_solr
[junit4] 2> 762608 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 762608 DEBUG (ScheduledTrigger-1330-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 762608 DEBUG (ScheduledTrigger-1330-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10001_solr
[junit4] 2> 762647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 762648 DEBUG (ScheduledTrigger-1330-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"3d2e48590d0c6eTbthhb7oewwdldqsdeqsexzo9u",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":17220861845900398,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[17220861845900398],
[junit4] 2> "_dequeue_time_":17220891839520548,
[junit4] 2> "_enqueue_time_":17220863899672698,
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 763079 DEBUG (ScheduledTrigger-1330-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 763081 DEBUG (AutoscalingActionExecutor-1328-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 44809 ms for event id=3d2e48590d0c6eTbthhb7oewwdldqsdeqsexzo9u
[junit4] 2> 763147 DEBUG (ScheduledTrigger-1330-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 763147 DEBUG (ScheduledTrigger-1330-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 763148 DEBUG (AutoscalingActionExecutor-1331-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"3d2e48590d0c6eTbthhb7oewwdldqsdeqsexzo9u",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":17220861845900398,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[17220861845900398],
[junit4] 2> "_dequeue_time_":17220891839520548,
[junit4] 2> "_enqueue_time_":17220863899672698,
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 763148 INFO (AutoscalingActionExecutor-1331-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"3d2e48590d0c6eTbthhb7oewwdldqsdeqsexzo9u",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":17220861845900398,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[17220861845900398],
[junit4] 2> "_dequeue_time_":17220891839520548,
[junit4] 2> "_enqueue_time_":17220863899672698,
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 763167 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEventQueue
[junit4] 2> 763167 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 763167 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 763167 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 763167 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 763167 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 763167 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 2
[junit4] 2> 763167 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 1
[junit4] 2> 763168 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10001_solr
[junit4] 2> 763168 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 763168 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 763168 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 763168 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 763187 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 763189 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 1
[junit4] 2> 763190 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 763190 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 763190 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger1
[junit4] 2> 763191 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 2
[junit4] 2> 763468 DEBUG (AutoscalingActionExecutor-1331-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 763468 DEBUG (AutoscalingActionExecutor-1331-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 763468 DEBUG (AutoscalingActionExecutor-1331-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 16024 ms for event id=3d2e48590d0c6eTbthhb7oewwdldqsdeqsexzo9u
[junit4] 2> 763471 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeMarkersRegistration
[junit4] 2> 763471 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 763471 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10001_solr
[junit4] 2> 763471 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 763471 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 763472 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 763475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 763475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 763475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
[junit4] 2> 763475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion -1
[junit4] 2> 763475 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 763486 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 763486 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 763486 DEBUG (ScheduledTrigger-1333-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 763514 DEBUG (ScheduledTrigger-1333-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 763559 DEBUG (ScheduledTrigger-1333-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 763584 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== KILL OVERSEER 1
[junit4] 2> 763584 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 763604 DEBUG (ScheduledTrigger-1333-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 763604 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 763640 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 763669 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 763669 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 763670 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion -1
[junit4] 2> 763670 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 763670 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10000_solr
[junit4] 2> 763670 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 763670 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 763670 DEBUG (ScheduledTrigger-1336-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 763670 DEBUG (ScheduledTrigger-1336-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10000_solr
[junit4] 2> 763703 DEBUG (ScheduledTrigger-1336-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 763783 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== ADD TRIGGERS
[junit4] 2> 763936 DEBUG (ScheduledTrigger-1336-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 763940 DEBUG (simCloudManagerPool-1335-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 763941 DEBUG (simCloudManagerPool-1335-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 763941 DEBUG (simCloudManagerPool-1335-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 763941 DEBUG (simCloudManagerPool-1335-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 763941 DEBUG (simCloudManagerPool-1335-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventMarkerAction}], enabled=true}
[junit4] 2> 763941 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 763941 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 763941 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 763954 DEBUG (ScheduledTrigger-1336-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 763954 DEBUG (ScheduledTrigger-1336-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 763964 DEBUG (simCloudManagerPool-1335-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 763964 DEBUG (simCloudManagerPool-1335-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 763964 DEBUG (simCloudManagerPool-1335-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 763964 DEBUG (simCloudManagerPool-1335-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 763964 DEBUG (simCloudManagerPool-1335-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10002_solr, 127.0.0.1:10003_solr]
[junit4] 2> 763965 DEBUG (simCloudManagerPool-1335-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventMarkerAction}], enabled=true}
[junit4] 2> 763971 DEBUG (ScheduledTrigger-1336-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 763983 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 763983 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 763983 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 763983 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== ADD NODE 1
[junit4] 2> 763984 DEBUG (ScheduledTrigger-1336-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 763985 DEBUG (ScheduledTrigger-1336-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 763985 DEBUG (ScheduledTrigger-1336-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 763985 DEBUG (ScheduledTrigger-1336-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10004_solr at time 17220958697168748
[junit4] 2> 764011 DEBUG (ScheduledTrigger-1336-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 764011 DEBUG (ScheduledTrigger-1336-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 764011 DEBUG (ScheduledTrigger-1336-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 764011 DEBUG (ScheduledTrigger-1336-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 764011 DEBUG (ScheduledTrigger-1336-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10004_solr] added at times [17220958697168748], now=17220960030893748
[junit4] 2> 764011 DEBUG (ScheduledTrigger-1336-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"3d2e5ee5d61f6cTbthhb7oewwdldqsdeqsexzo9y",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":17220958697168748,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[17220958697168748],
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 764012 DEBUG (ScheduledTrigger-1336-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger, node_added_trigger]
[junit4] 2> 764067 DEBUG (AutoscalingActionExecutor-1337-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"3d2e5ee5d61f6cTbthhb7oewwdldqsdeqsexzo9y",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":17220958697168748,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[17220958697168748],
[junit4] 2> "_enqueue_time_":17220960083099548,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 764091 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== KILL OVERSEER 2
[junit4] 2> 764091 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 764091 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 764092 WARN (AutoscalingActionExecutor-1337-thread-1) [ ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
[junit4] 2> java.io.IOException: java.lang.InterruptedException
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:462)
[junit4] 2> at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:122)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:745)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:715)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:313)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404)
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:191)
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:460)
[junit4] 2> ... 10 more
[junit4] 2> 764092 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A95B73A61123EBCC]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 764093 DEBUG (AutoscalingActionExecutor-1337-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 1308 ms for event id=3d2e5ee5d61f6cTbthhb7oewwdldqsdeqsexzo9y
[junit4] 2> 764094 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 764094 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10002_solr]
[junit4] 2> 764094 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10002_solr]
[junit4] 2> 764094 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10002_solr]
[junit4] 2> 764094 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventMarkerAction}], enabled=true}
[junit4] 2> 764094 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 764094 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 764094 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 764094 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10003_solr
[junit4] 2> 764103 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 764107 DEBUG (ScheduledTrigger-1339-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 764107 DEBUG (ScheduledTrigger-1339-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10003_solr
[junit4] 2> 764107 DEBUG (ScheduledTrigger-1339-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 764107 DEBUG (ScheduledTrigger-1339-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 764107 DEBUG (ScheduledTrigger-1339-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 764108 DEBUG (ScheduledTrigger-1339-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10003_solr
[junit4] 2> 764115 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 764135 DEBUG (ScheduledTrigger-1339-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 764135 DEBUG (ScheduledTrigger-1339-thread-2) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrig
[...truncated too long message...]
ave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/bcprov-jdk15on-1.54.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/commons-math3-3.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/analysis/icu/lucene-analyzers-icu-7.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/contrib/analysis-extras/lib/icu4j-60.2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180126_120144_4552991386129405292756.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180126_120144_4556646011001255056730.suites -stdin
[junit4] ERROR: JVM J2 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180126_120144_4554353534073021477340.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J2-20180126_120144_4556110147333232565113.syserr
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
[junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] at java.lang.Thread.run(Thread.java:748)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:836: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:780: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build.xml:262: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/common-build.xml:556: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:1513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:1038: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180126_120144_4551559829363305898675.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180126_120144_455213708881078249648.syserr
Total time: 321 minutes 6 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-7.x #331 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-7.x - Build # 333 - Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/333/
4 tests failed.
FAILED: org.apache.solr.cloud.AddReplicaTest.test
Error Message:
core_node6:{"core":"addreplicatest_coll_shard1_replica_n5","base_url":"https://127.0.0.1:58992/solr","node_name":"127.0.0.1:58992_solr","state":"active","type":"NRT"}
Stack Trace:
java.lang.AssertionError: core_node6:{"core":"addreplicatest_coll_shard1_replica_n5","base_url":"https://127.0.0.1:58992/solr","node_name":"127.0.0.1:58992_solr","state":"active","type":"NRT"}
at __randomizedtesting.SeedInfo.seed([12FFC1B728E5149A:9AABFE6D86197962]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.AddReplicaTest.test(AddReplicaTest.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestPolicyCloud.testCreateCollectionAddReplica
Error Message:
java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException: org.apache.solr.common.SolrException: Error getting replica locations
Stack Trace:
java.io.IOException: java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException: org.apache.solr.common.SolrException: Error getting replica locations
at __randomizedtesting.SeedInfo.seed([12FFC1B728E5149A:92DFA49939A6FC3C]:0)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:462)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager$1.request(SimCloudManager.java:336)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.autoscaling.sim.TestPolicyCloud.testCreateCollectionAddReplica(TestPolicyCloud.java:123)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: org.apache.solr.common.SolrException: org.apache.solr.common.SolrException: Error getting replica locations
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:460)
... 44 more
Caused by: org.apache.solr.common.SolrException: org.apache.solr.common.SolrException: Error getting replica locations
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simHandleSolrRequest(SimCloudManager.java:572)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.lambda$request$2(SimCloudManager.java:459)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.solr.common.SolrException: Error getting replica locations
at org.apache.solr.cloud.api.collections.Assign.getPositionsUsingPolicy(Assign.java:400)
at org.apache.solr.cloud.api.collections.Assign.identifyNodes(Assign.java:296)
at org.apache.solr.cloud.api.collections.AddReplicaCmd.assignReplicaDetails(AddReplicaCmd.java:239)
at org.apache.solr.cloud.autoscaling.sim.SimClusterStateProvider.simAddReplica(SimClusterStateProvider.java:333)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simHandleSolrRequest(SimCloudManager.java:570)
... 6 more
Caused by: org.apache.solr.common.SolrException: No node can satisfy the rules [{
"replica":0,
"shard":"#EACH",
"port":"!10022",
"collection":"testCreateCollectionAddReplica"}]
at org.apache.solr.client.solrj.cloud.autoscaling.PolicyHelper.getReplicaLocations(PolicyHelper.java:162)
at org.apache.solr.cloud.api.collections.Assign.getPositionsUsingPolicy(Assign.java:388)
... 10 more
FAILED: org.apache.solr.security.TestPKIAuthenticationPlugin.test
Error Message:
Stack Trace:
java.lang.NullPointerException
at __randomizedtesting.SeedInfo.seed([12FFC1B728E5149A:9AABFE6D86197962]:0)
at org.apache.solr.security.TestPKIAuthenticationPlugin.test(TestPKIAuthenticationPlugin.java:102)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([12FFC1B728E5149A:4FB7DF3EE723B2D5]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12486 lines...]
[junit4] Suite: org.apache.solr.security.TestPKIAuthenticationPlugin
[junit4] 2> 3151415 INFO (SUITE-TestPKIAuthenticationPlugin-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.security.TestPKIAuthenticationPlugin_12FFC1B728E5149A-001/init-core-data-001
[junit4] 2> 3151421 WARN (SUITE-TestPKIAuthenticationPlugin-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=56 numCloses=56
[junit4] 2> 3151421 INFO (SUITE-TestPKIAuthenticationPlugin-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 3151567 INFO (SUITE-TestPKIAuthenticationPlugin-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 3151595 INFO (TEST-TestPKIAuthenticationPlugin.test-seed#[12FFC1B728E5149A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 3176169 ERROR (TEST-TestPKIAuthenticationPlugin.test-seed#[12FFC1B728E5149A]) [ ] o.a.s.s.PKIAuthenticationPlugin Invalid key request timestamp: 1516925494047 , received timestamp: 1516925516049 , TTL: 10000
[junit4] 2> 3176170 INFO (TEST-TestPKIAuthenticationPlugin.test-seed#[12FFC1B728E5149A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending test
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestPKIAuthenticationPlugin -Dtests.method=test -Dtests.seed=12FFC1B728E5149A -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=es -Dtests.timezone=PNT -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 24.7s J2 | TestPKIAuthenticationPlugin.test <<<
[junit4] > Throwable #1: java.lang.NullPointerException
[junit4] > at __randomizedtesting.SeedInfo.seed([12FFC1B728E5149A:9AABFE6D86197962]:0)
[junit4] > at org.apache.solr.security.TestPKIAuthenticationPlugin.test(TestPKIAuthenticationPlugin.java:102)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.security.TestPKIAuthenticationPlugin_12FFC1B728E5149A-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=932, maxMBSortInHeap=7.003191279414514, sim=RandomSimilarity(queryNorm=true): {}, locale=es, timezone=PNT
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=159850704,total=532676608
[junit4] 2> NOTE: All tests run in this JVM: [TestSolrCoreParser, RecoveryAfterSoftCommitTest, TestCoreContainer, TestFastWriter, TestBadConfig, TestRandomDVFaceting, ManagedSchemaRoundRobinCloudTest, ReplicationFactorTest, CollectionTooManyReplicasTest, TestNumericTokenStream, TestLMJelinekMercerSimilarityFactory, TestSuggestSpellingConverter, TestDefaultStatsCache, TestEmbeddedSolrServerConstructors, CopyFieldTest, CoreAdminHandlerTest, TestSolrConfigHandlerCloud, TestStressReorder, DistributedFacetPivotWhiteBoxTest, AnalyticsMergeStrategyTest, TestSolrCoreSnapshots, TestFieldTypeCollectionResource, TestAnalyzedSuggestions, AnalysisErrorHandlingTest, FileBasedSpellCheckerTest, HdfsChaosMonkeyNothingIsSafeTest, CdcrReplicationDistributedZkTest, TestXmlQParserPlugin, DeleteNodeTest, AutoAddReplicasPlanActionTest, TestBinaryResponseWriter, ShardSplitTest, TestNumericTerms64, OverseerStatusTest, SparseHLLTest, TestManagedResource, TestSolrConfigHandler, TestPKIAuthenticationPlugin]
[junit4] Completed [152/767 (1!)] on J2 in 25.02s, 1 test, 1 error <<< FAILURES!
[...truncated 1 lines...]
[junit4] JVM J1: stdout was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180125_231832_0127675485762864369299.sysout
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] #
[junit4] # There is insufficient memory for the Java Runtime Environment to continue.
[junit4] # Native memory allocation (mmap) failed to map 12288 bytes for committing reserved memory.
[junit4] # An error report file with more information is saved as:
[junit4] # /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J1/hs_err_pid6549.log
[junit4] <<< JVM J1: EOF ----
[junit4] JVM J1: stderr was not empty, see: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180125_231832_012916672818556303816.syserr
[junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
[junit4] Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00007f3a61d0f000, 12288, 0) failed; error='Cannot allocate memory' (errno=12)
[junit4] <<< JVM J1: EOF ----
[...truncated 1015 lines...]
[junit4] Suite: org.apache.solr.cloud.AddReplicaTest
[junit4] 2> 8002901 INFO (SUITE-AddReplicaTest-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.AddReplicaTest_12FFC1B728E5149A-001/init-core-data-001
[junit4] 2> 8002902 WARN (SUITE-AddReplicaTest-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 8002902 INFO (SUITE-AddReplicaTest-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 8002908 INFO (SUITE-AddReplicaTest-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 8002909 INFO (SUITE-AddReplicaTest-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.AddReplicaTest_12FFC1B728E5149A-001/tempDir-001
[junit4] 2> 8002909 INFO (SUITE-AddReplicaTest-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 8002920 INFO (Thread-3192) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 8002921 INFO (Thread-3192) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 8002953 ERROR (Thread-3192) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 8003020 INFO (SUITE-AddReplicaTest-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:48539
[junit4] 2> 8003026 INFO (zkConnectionManagerCallback-4468-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8003062 INFO (jetty-launcher-4465-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 8003062 INFO (jetty-launcher-4465-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 8003110 INFO (jetty-launcher-4465-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 8003110 INFO (jetty-launcher-4465-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 8003110 INFO (jetty-launcher-4465-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 8003110 INFO (jetty-launcher-4465-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@72817e2{/solr,null,AVAILABLE}
[junit4] 2> 8003111 INFO (jetty-launcher-4465-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@767ba0ce{SSL,[ssl, http/1.1]}{127.0.0.1:58992}
[junit4] 2> 8003111 INFO (jetty-launcher-4465-thread-1) [ ] o.e.j.s.Server Started @8030697ms
[junit4] 2> 8003111 INFO (jetty-launcher-4465-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58992}
[junit4] 2> 8003111 ERROR (jetty-launcher-4465-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 8003112 INFO (jetty-launcher-4465-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 8003112 INFO (jetty-launcher-4465-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 8003112 INFO (jetty-launcher-4465-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 8003112 INFO (jetty-launcher-4465-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-26T01:32:23.120Z
[junit4] 2> 8003129 INFO (jetty-launcher-4465-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 8003129 INFO (jetty-launcher-4465-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 8003129 INFO (jetty-launcher-4465-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 8003130 INFO (jetty-launcher-4465-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b30762{/solr,null,AVAILABLE}
[junit4] 2> 8003130 INFO (jetty-launcher-4465-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@77231328{SSL,[ssl, http/1.1]}{127.0.0.1:45477}
[junit4] 2> 8003130 INFO (jetty-launcher-4465-thread-2) [ ] o.e.j.s.Server Started @8030716ms
[junit4] 2> 8003130 INFO (jetty-launcher-4465-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45477}
[junit4] 2> 8003130 ERROR (jetty-launcher-4465-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 8003130 INFO (jetty-launcher-4465-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 8003131 INFO (jetty-launcher-4465-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 8003131 INFO (jetty-launcher-4465-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 8003131 INFO (jetty-launcher-4465-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-26T01:32:23.139Z
[junit4] 2> 8003221 INFO (jetty-launcher-4465-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 8003222 INFO (jetty-launcher-4465-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 8003224 INFO (zkConnectionManagerCallback-4470-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8003225 INFO (jetty-launcher-4465-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 8003286 INFO (jetty-launcher-4465-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48539/solr
[junit4] 2> 8003320 INFO (jetty-launcher-4465-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 8003325 INFO (jetty-launcher-4465-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 8003325 INFO (jetty-launcher-4465-thread-4) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 8003321 INFO (zkConnectionManagerCallback-4476-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8003321 INFO (zkConnectionManagerCallback-4472-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8003326 INFO (jetty-launcher-4465-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 8003321 INFO (jetty-launcher-4465-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 8003330 INFO (jetty-launcher-4465-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 8003330 INFO (jetty-launcher-4465-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 8003365 INFO (jetty-launcher-4465-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@20413aa1{/solr,null,AVAILABLE}
[junit4] 2> 8003366 INFO (jetty-launcher-4465-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@14a2ddaa{SSL,[ssl, http/1.1]}{127.0.0.1:55453}
[junit4] 2> 8003366 INFO (jetty-launcher-4465-thread-4) [ ] o.e.j.s.Server Started @8030952ms
[junit4] 2> 8003366 INFO (jetty-launcher-4465-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55453}
[junit4] 2> 8003366 ERROR (jetty-launcher-4465-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 8003367 INFO (jetty-launcher-4465-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 8003367 INFO (jetty-launcher-4465-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 8003367 INFO (jetty-launcher-4465-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 8003367 INFO (jetty-launcher-4465-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-26T01:32:23.375Z
[junit4] 2> 8003493 INFO (jetty-launcher-4465-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1156340b{/solr,null,AVAILABLE}
[junit4] 2> 8003494 INFO (jetty-launcher-4465-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4e2d3343{SSL,[ssl, http/1.1]}{127.0.0.1:58745}
[junit4] 2> 8003494 INFO (jetty-launcher-4465-thread-3) [ ] o.e.j.s.Server Started @8031080ms
[junit4] 2> 8003494 INFO (jetty-launcher-4465-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58745}
[junit4] 2> 8003494 ERROR (jetty-launcher-4465-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 8003509 INFO (jetty-launcher-4465-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.3.0
[junit4] 2> 8003509 INFO (jetty-launcher-4465-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 8003509 INFO (jetty-launcher-4465-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 8003509 INFO (jetty-launcher-4465-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-26T01:32:23.517Z
[junit4] 2> 8003529 INFO (zkConnectionManagerCallback-4480-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8003529 INFO (jetty-launcher-4465-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 8003563 INFO (zkConnectionManagerCallback-4478-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8003566 DEBUG (jetty-launcher-4465-thread-1) [n:127.0.0.1:58992_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$39/181677484@262a6df9
[junit4] 2> 8003711 INFO (jetty-launcher-4465-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 8003738 INFO (jetty-launcher-4465-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48539/solr
[junit4] 2> 8003739 INFO (zkConnectionManagerCallback-4487-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8003788 INFO (jetty-launcher-4465-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48539/solr
[junit4] 2> 8003801 DEBUG (jetty-launcher-4465-thread-2) [n:127.0.0.1:45477_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$39/181677484@29147ce2
[junit4] 2> 8003872 INFO (zkConnectionManagerCallback-4490-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8003873 INFO (zkConnectionManagerCallback-4482-thread-1-processing-n:127.0.0.1:58992_solr) [n:127.0.0.1:58992_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8003905 INFO (jetty-launcher-4465-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:48539/solr
[junit4] 2> 8003978 DEBUG (jetty-launcher-4465-thread-3) [n:127.0.0.1:58745_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$39/181677484@35608264
[junit4] 2> 8004001 INFO (zkConnectionManagerCallback-4492-thread-1-processing-n:127.0.0.1:45477_solr) [n:127.0.0.1:45477_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8004002 INFO (zkConnectionManagerCallback-4496-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8004002 INFO (zkConnectionManagerCallback-4499-thread-1-processing-n:127.0.0.1:58745_solr) [n:127.0.0.1:58745_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8004048 DEBUG (jetty-launcher-4465-thread-4) [n:127.0.0.1:55453_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$39/181677484@68f59d12
[junit4] 2> 8004049 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 8004049 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004060 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 8004060 INFO (zkConnectionManagerCallback-4503-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8004061 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 8004061 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 8004068 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 8004069 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004084 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 8004085 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 8004089 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 8004104 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 8004105 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004105 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 8004105 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 8004105 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 8004106 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 8004108 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 8004209 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 8004210 INFO (jetty-launcher-4465-thread-4) [n:127.0.0.1:55453_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 8004211 INFO (jetty-launcher-4465-thread-1) [n:127.0.0.1:58992_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 8004211 DEBUG (jetty-launcher-4465-thread-1) [n:127.0.0.1:58992_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/99413546954260486-127.0.0.1:58992_solr-n_0000000001
[junit4] 2> 8004211 DEBUG (jetty-launcher-4465-thread-4) [n:127.0.0.1:55453_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/99413546954260492-127.0.0.1:55453_solr-n_0000000000
[junit4] 2> 8004211 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/99413546954260492-127.0.0.1:55453_solr-n_0000000000
[junit4] 2> 8004212 DEBUG (jetty-launcher-4465-thread-1) [n:127.0.0.1:58992_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/99413546954260492-127.0.0.1:55453_solr-n_0000000000 to know if I could be the leader
[junit4] 2> 8004212 INFO (jetty-launcher-4465-thread-4) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55453_solr
[junit4] 2> 8004213 INFO (jetty-launcher-4465-thread-4) [n:127.0.0.1:55453_solr ] o.a.s.c.Overseer Overseer (id=99413546954260492-127.0.0.1:55453_solr-n_0000000000) starting
[junit4] 2> 8004227 INFO (jetty-launcher-4465-thread-1) [n:127.0.0.1:58992_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58992_solr
[junit4] 2> 8004236 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8004237 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8004237 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8004257 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004261 INFO (zkCallback-4481-thread-1-processing-n:127.0.0.1:58992_solr) [n:127.0.0.1:58992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 8004265 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004277 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004280 INFO (jetty-launcher-4465-thread-3) [n:127.0.0.1:58745_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 8004298 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 8004300 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 8004409 INFO (zkCallback-4502-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 8004409 INFO (zkCallback-4498-thread-1-processing-n:127.0.0.1:58745_solr) [n:127.0.0.1:58745_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 8004508 DEBUG (jetty-launcher-4465-thread-3) [n:127.0.0.1:58745_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/99413546954260491-127.0.0.1:58745_solr-n_0000000002
[junit4] 2> 8004510 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/99413546954260486-127.0.0.1:58992_solr-n_0000000001
[junit4] 2> 8004510 DEBUG (jetty-launcher-4465-thread-3) [n:127.0.0.1:58745_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/99413546954260486-127.0.0.1:58992_solr-n_0000000001 to know if I could be the leader
[junit4] 2> 8004512 DEBUG (jetty-launcher-4465-thread-4) [n:127.0.0.1:55453_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:55453_solr as DOWN
[junit4] 2> 8004514 DEBUG (OverseerAutoScalingTriggerThread-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 8004528 DEBUG (jetty-launcher-4465-thread-3) [n:127.0.0.1:58745_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:58745_solr as DOWN
[junit4] 2> 8004545 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/autoscaling.json
[junit4] 2> 8004552 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 8004552 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 8004553 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 8004553 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 8004553 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 8004553 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 8004553 INFO (jetty-launcher-4465-thread-4) [n:127.0.0.1:55453_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55453_solr
[junit4] 2> 8004553 DEBUG (OverseerAutoScalingTriggerThread-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 8004553 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 8004560 DEBUG (OverseerAutoScalingTriggerThread-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:58992_solr]
[junit4] 2> 8004560 DEBUG (OverseerAutoScalingTriggerThread-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 8004561 DEBUG (OverseerAutoScalingTriggerThread-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 8004566 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8004566 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8004581 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8004582 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004589 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004589 INFO (zkCallback-4498-thread-1-processing-n:127.0.0.1:58745_solr) [n:127.0.0.1:58745_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 8004589 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004591 INFO (zkCallback-4481-thread-1-processing-n:127.0.0.1:58992_solr) [n:127.0.0.1:58992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 8004609 INFO (zkCallback-4502-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 8004609 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue-work
[junit4] 2> 8004709 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8004709 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 8004710 INFO (jetty-launcher-4465-thread-3) [n:127.0.0.1:58745_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58745_solr
[junit4] 2> 8004710 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:55453_solr"} current state version: 0
[junit4] 2> 8004710 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:55453_solr
[junit4] 2> 8004710 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8004711 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8004711 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8004712 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8004712 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004712 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004712 INFO (zkCallback-4481-thread-1-processing-n:127.0.0.1:58992_solr) [n:127.0.0.1:58992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 8004716 INFO (zkCallback-4502-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 8004717 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:58745_solr"} current state version: 0
[junit4] 2> 8004717 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:58745_solr
[junit4] 2> 8004720 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8004728 INFO (zkCallback-4498-thread-1-processing-n:127.0.0.1:58745_solr) [n:127.0.0.1:58745_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 8004829 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 8004869 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8005280 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 8005281 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8005281 INFO (jetty-launcher-4465-thread-2) [n:127.0.0.1:45477_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 8005281 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 8005284 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 8005285 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 8005289 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 8005305 INFO (jetty-launcher-4465-thread-2) [n:127.0.0.1:45477_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 8005305 DEBUG (jetty-launcher-4465-thread-2) [n:127.0.0.1:45477_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/99413546954260489-127.0.0.1:45477_solr-n_0000000003
[junit4] 2> 8005306 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/99413546954260491-127.0.0.1:58745_solr-n_0000000002
[junit4] 2> 8005306 DEBUG (jetty-launcher-4465-thread-2) [n:127.0.0.1:45477_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/99413546954260491-127.0.0.1:58745_solr-n_0000000002 to know if I could be the leader
[junit4] 2> 8005306 DEBUG (jetty-launcher-4465-thread-2) [n:127.0.0.1:45477_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:45477_solr as DOWN
[junit4] 2> 8005307 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 8005308 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8005308 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:45477_solr"} current state version: 0
[junit4] 2> 8005308 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:45477_solr
[junit4] 2> 8005309 INFO (jetty-launcher-4465-thread-2) [n:127.0.0.1:45477_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45477_solr
[junit4] 2> 8005309 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8005309 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8005325 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8005325 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 8005325 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8005325 INFO (zkCallback-4502-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 8005326 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8005326 INFO (zkCallback-4481-thread-1-processing-n:127.0.0.1:58992_solr) [n:127.0.0.1:58992_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 8005333 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8005333 INFO (zkCallback-4498-thread-1-processing-n:127.0.0.1:58745_solr) [n:127.0.0.1:58745_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 8005372 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8005409 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 8005410 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8005796 DEBUG (OverseerAutoScalingTriggerThread-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 8005869 INFO (zkCallback-4491-thread-1-processing-n:127.0.0.1:45477_solr) [n:127.0.0.1:45477_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 8005920 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 8005921 DEBUG (OverseerAutoScalingTriggerThread-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 8005921 DEBUG (ScheduledTrigger-6351-thread-1) [n:127.0.0.1:55453_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 8006281 INFO (jetty-launcher-4465-thread-1) [n:127.0.0.1:58992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58992.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8006326 INFO (jetty-launcher-4465-thread-3) [n:127.0.0.1:58745_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58745.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8006529 INFO (jetty-launcher-4465-thread-1) [n:127.0.0.1:58992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58992.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8006529 INFO (jetty-launcher-4465-thread-1) [n:127.0.0.1:58992_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58992.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8006558 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 8006558 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 8006558 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 8006559 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 8006559 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 8006559 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 8006559 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 8006592 INFO (jetty-launcher-4465-thread-1) [n:127.0.0.1:58992_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.AddReplicaTest_12FFC1B728E5149A-001/tempDir-001/node1/.
[junit4] 2> 8006928 DEBUG (ScheduledTrigger-6351-thread-1) [n:127.0.0.1:55453_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 8007212 INFO (jetty-launcher-4465-thread-3) [n:127.0.0.1:58745_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58745.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8007216 INFO (jetty-launcher-4465-thread-4) [n:127.0.0.1:55453_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55453.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8007220 INFO (jetty-launcher-4465-thread-3) [n:127.0.0.1:58745_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58745.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8007222 INFO (jetty-launcher-4465-thread-3) [n:127.0.0.1:58745_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.AddReplicaTest_12FFC1B728E5149A-001/tempDir-001/node3/.
[junit4] 2> 8007272 INFO (jetty-launcher-4465-thread-4) [n:127.0.0.1:55453_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55453.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8007325 INFO (jetty-launcher-4465-thread-4) [n:127.0.0.1:55453_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55453.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8007348 INFO (jetty-launcher-4465-thread-4) [n:127.0.0.1:55453_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.AddReplicaTest_12FFC1B728E5149A-001/tempDir-001/node4/.
[junit4] 2> 8007754 INFO (jetty-launcher-4465-thread-2) [n:127.0.0.1:45477_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45477.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8007863 INFO (jetty-launcher-4465-thread-2) [n:127.0.0.1:45477_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45477.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8007864 INFO (jetty-launcher-4465-thread-2) [n:127.0.0.1:45477_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45477.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4c790975
[junit4] 2> 8007888 INFO (jetty-launcher-4465-thread-2) [n:127.0.0.1:45477_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.AddReplicaTest_12FFC1B728E5149A-001/tempDir-001/node2/.
[junit4] 2> 8008009 DEBUG (ScheduledTrigger-6351-thread-2) [n:127.0.0.1:55453_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 8008580 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor Got 0 tasks from work-queue : [[]]
[junit4] 2> 8008581 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 8008581 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 8008581 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 8008581 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 8008581 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 8008581 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 8008785 INFO (zkConnectionManagerCallback-4510-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8008828 INFO (zkConnectionManagerCallback-4514-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8008828 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterprops.json
[junit4] 2> 8008829 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 8008829 INFO (SUITE-AddReplicaTest-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 8008829 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 8008844 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 8008845 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 8008845 INFO (SUITE-AddReplicaTest-seed#[12FFC1B728E5149A]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:48539/solr ready
[junit4] 2> 8009016 DEBUG (ScheduledTrigger-6351-thread-1) [n:127.0.0.1:55453_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 8009328 INFO (TEST-AddReplicaTest.test-seed#[12FFC1B728E5149A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 8009338 INFO (zkConnectionManagerCallback-4516-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 8009514 INFO (qtp834208081-17373) [n:127.0.0.1:55453_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&maxShardsPerNode=2&name=addreplicatest_coll&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 8009785 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
[junit4] 2> 8009785 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/overseer/collection-queue-work/qnr-0000000000
[junit4] 2> 8009785 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 8009786 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
[junit4] 2> 8009786 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
[junit4] 2> 8009786 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 8009804 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
[junit4] 2> 8009805 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskQueue Returning topN elements: [/overseer/collection-queue-work/qn-0000000000, ]
[junit4] 2> 8009805 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor Got 1 tasks from work-queue : [[org.apache.solr.cloud.OverseerTaskQueue$QueueEvent@2e684ecf]]
[junit4] 2> 8009805 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000000] as running
[junit4] 2> 8009805 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "name":"addreplicatest_coll",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"}
[junit4] 2> 8009808 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 1
[junit4] 2> 8009808 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor RunningTasks: [/overseer/collection-queue-work/qn-0000000000]
[junit4] 2> 8009808 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 8009808 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 8009808 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000000]
[junit4] 2> 8009808 DEBUG (OverseerCollectionConfigSetProcessor-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskQueue Peeking for top 99 elements. ExcludeSet: [/overseer/collection-queue-work/qn-0000000000],[]
[junit4] 2> 8009808 DEBUG (OverseerThreadFactory-6353-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.OverseerTaskProcessor Runner processing /overseer/collection-queue-work/qn-0000000000
[junit4] 2> 8009808 DEBUG (OverseerThreadFactory-6353-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler OverseerCollectionMessageHandler.processMessage : create , {
[junit4] 2> "name":"addreplicatest_coll",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"}
[junit4] 2> 8009808 INFO (OverseerThreadFactory-6353-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection addreplicatest_coll
[junit4] 2> 8009814 DEBUG (OverseerThreadFactory-6353-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.a.c.Assign Identify nodes using default
[junit4] 2> 8009814 DEBUG (OverseerThreadFactory-6353-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler creating collections conf node /collections/addreplicatest_coll
[junit4] 2> 8009826 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 8009826 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 8009826 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 8009826 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 8009826 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 8009828 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 8009829 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 8009859 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 8009859 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 8009859 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 8009865 DEBUG (OverseerThreadFactory-6353-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.a.c.CreateCollectionCmd Check for collection zkNode:addreplicatest_coll
[junit4] 2> 8009880 DEBUG (OverseerThreadFactory-6353-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.a.c.CreateCollectionCmd Collection zkNode exists
[junit4] 2> 8009882 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 8009928 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8009929 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"addreplicatest_coll",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 8009929 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.ClusterStateMutator building a new cName: addreplicatest_coll
[junit4] 2> 8009929 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/addreplicatest_coll/state.json
[junit4] 2> 8009930 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 8009930 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8009989 DEBUG (OverseerThreadFactory-6353-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.a.c.CreateCollectionCmd Creating SolrCores for new collection addreplicatest_coll, shardNames [shard1, shard2] , message : {
[junit4] 2> "name":"addreplicatest_coll",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"}
[junit4] 2> 8009991 DEBUG (OverseerThreadFactory-6353-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.a.c.CreateCollectionCmd Creating core addreplicatest_coll_shard1_replica_n1 as part of shard shard1 of collection addreplicatest_coll on 127.0.0.1:45477_solr
[junit4] 2> 8009992 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 8009992 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8009993 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45477/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 8009993 INFO (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45477/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 8010008 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 8010008 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node2":{
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:45477/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 8010009 DEBUG (OverseerThreadFactory-6353-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.a.c.CreateCollectionCmd Creating core addreplicatest_coll_shard2_replica_n3 as part of shard shard2 of collection addreplicatest_coll on 127.0.0.1:55453_solr
[junit4] 2> 8010010 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 8010011 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8010011 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:55453/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 8010011 INFO (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:55453/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 8010012 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.SliceMutator Old Slice: shard2:{
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 8010012 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.SliceMutator New Slice: shard2:{
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node4":{
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n3",
[junit4] 2> "base_url":"https://127.0.0.1:55453/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 8010029 DEBUG (OverseerThreadFactory-6353-thread-1-processing-n:127.0.0.1:55453_solr) [n:127.0.0.1:55453_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Expecting [addreplicatest_coll_shard1_replica_n1, addreplicatest_coll_shard2_replica_n3] cores but found {}
[junit4] 2> 8010036 DEBUG (ScheduledTrigger-6351-thread-1) [n:127.0.0.1:55453_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 8010121 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/addreplicatest_coll/state.json version: 0
[junit4] 2> 8010121 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 8010122 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8010375 INFO (qtp834208081-17370) [n:127.0.0.1:55453_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&newCollection=true&name=addreplicatest_coll_shard2_replica_n3&action=CREATE&numShards=2&collection=addreplicatest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 8010376 INFO (qtp834208081-17370) [n:127.0.0.1:55453_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 8010385 INFO (qtp1281864522-17357) [n:127.0.0.1:45477_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=addreplicatest_coll_shard1_replica_n1&action=CREATE&numShards=2&collection=addreplicatest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 8010385 INFO (qtp1281864522-17357) [n:127.0.0.1:45477_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 8010394 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 8010394 DEBUG (qtp1281864522-17357) [n:127.0.0.1:45477_solr c:addreplicatest_coll s:shard1 r:core_node2 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ZkController publishing state=down
[junit4] 2> 8010395 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 8010395 DEBUG (qtp1281864522-17357) [n:127.0.0.1:45477_solr c:addreplicatest_coll s:shard1 r:core_node2 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ZkController Registering watch for collection addreplicatest_coll
[junit4] 2> 8010396 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 8010396 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8010396 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:45477/solr",
[junit4] 2> "node_name":"127.0.0.1:45477_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 8010397 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:45477/solr",
[junit4] 2> "node_name":"127.0.0.1:45477_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 8010397 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node2:{"core":"addreplicatest_coll_shard1_replica_n1","base_url":"https://127.0.0.1:45477/solr","node_name":"127.0.0.1:45477_solr","state":"down","type":"NRT"}
[junit4] 2> 8010397 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(addreplicatest_coll//collections/addreplicatest_coll/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{
[junit4] 2> "shard1":{
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node2":{
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "base_url":"https://127.0.0.1:45477/solr",
[junit4] 2> "node_name":"127.0.0.1:45477_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}},
[junit4] 2> "shard2":{
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node4":{
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n3",
[junit4] 2> "base_url":"https://127.0.0.1:55453/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 8010397 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 8010397 DEBUG (qtp1281864522-17357) [n:127.0.0.1:45477_solr c:addreplicatest_coll s:shard1 r:core_node2 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ZkController waiting to find shard id in clusterstate for addreplicatest_coll_shard1_replica_n1
[junit4] 2> 8010428 DEBUG (qtp834208081-17370) [n:127.0.0.1:55453_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n3] o.a.s.c.ZkController publishing state=down
[junit4] 2> 8010429 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 8010429 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 8010430 DEBUG (qtp834208081-17370) [n:127.0.0.1:55453_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n3] o.a.s.c.ZkController Registering watch for collection addreplicatest_coll
[junit4] 2> 8010430 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n3",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:55453/solr",
[junit4] 2> "node_name":"127.0.0.1:55453_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 8010430 DEBUG (OverseerStateUpdate-99413546954260492-127.0.0.1:55453_solr-n_0000000000) [n:127.0.0.1:55453_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n3",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"https://127.0.0.1:55453/solr",
[junit4] 2> "node_name":"127.0.0.1:55453_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"d
[...truncated too long message...]
ave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptor-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-admin-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-authn-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-authz-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-changelog-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-collective-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-event-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-exception-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-journal-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-normalization-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-operational-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-referral-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-schema-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-subtree-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-interceptors-trigger-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-jdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-jdbm1-2.0.0-M2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-kerberos-codec-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-ldif-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-mavibot-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-kerberos-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-ldap-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-protocol-shared-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/apacheds-xdbm-partition-2.0.0-M15.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/api-all-1.0.0-M20.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/bcprov-jdk15on-1.54.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/byte-buddy-1.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/commons-collections-3.2.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/commons-math3-3.6.1.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/ehcache-core-2.4.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-common-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-hdfs-2.7.4-tests.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/hadoop-minikdc-2.7.4.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/htrace-core-3.2.0-incubating.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jersey-core-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jersey-server-1.9.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-sslengine-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/jetty-util-6.1.26.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/mina-core-2.0.0-M5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/mockito-core-2.6.2.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/netty-all-4.0.36.Final.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/core/test-lib/objenesis-2.5.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/analysis/icu/lucene-analyzers-icu-7.3.0-SNAPSHOT.jar:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/contrib/solr-analysis-extras/classes/java:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/queryparser/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build/backward-codecs/classes/test:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/contrib/analysis-extras/lib/icu4j-60.2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-launcher.jar:/x1/jenkins/.ant/lib/ivy-2.4.0.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-log4j.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-junit4.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jai.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-javamail.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bsf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-net.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-antlr.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jsch.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-oro.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-commons-logging.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-netrexx.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-testutil.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jdepend.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-bcel.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-xalan2.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-resolver.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-jmf.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-apache-regexp.jar:/home/jenkins/tools/ant/apache-ant-1.8.4/lib/ant-swing.jar:/usr/local/asfpackages/java/jdk1.8.0_144/lib/tools.jar:/x1/jenkins/.ivy2/cache/com.carrotsearch.randomizedtesting/junit4-ant/jars/junit4-ant-2.5.3.jar com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe -eventsfile /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180125_231832_0121316788472702229222.events @/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180125_231832_0122514720359552545153.suites -stdin
[junit4] ERROR: JVM J1 ended with an exception: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180125_231832_0127675485762864369299.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180125_231832_012916672818556303816.syserr
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.executeSlave(JUnit4.java:1519)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4.access$000(JUnit4.java:126)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:982)
[junit4] at com.carrotsearch.ant.tasks.junit4.JUnit4$2.call(JUnit4.java:979)
[junit4] at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] at java.lang.Thread.run(Thread.java:748)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:836: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:780: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:59: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build.xml:262: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/common-build.xml:556: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:1513: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:1038: At least one slave process threw an exception, first: Forked process returned with error code: 1. Very likely a JVM crash. See process stdout at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180125_231832_0127675485762864369299.sysout See process stderr at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/temp/junit4-J1-20180125_231832_012916672818556303816.syserr
Total time: 399 minutes 48 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[Fast Archiver] No artifacts from Lucene-Solr-Tests-7.x #331 to compare, so performing full copy of artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-7.x - Build # 332 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-7.x/332/
3 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:37853_solr, 127.0.0.1:44062_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/18)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node3":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node3/data/", "base_url":"https://127.0.0.1:44062/solr", "node_name":"127.0.0.1:44062_solr", "type":"NRT", "ulogDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node3/data/tlog", "core":"testSimple1_shard1_replica_n1", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node5":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node5/data/", "base_url":"https://127.0.0.1:44062/solr", "node_name":"127.0.0.1:44062_solr", "type":"NRT", "ulogDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node5/data/tlog", "core":"testSimple1_shard1_replica_n2", "shared_storage":"true", "state":"active"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node7":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node7/data/", "base_url":"https://127.0.0.1:44062/solr", "node_name":"127.0.0.1:44062_solr", "type":"NRT", "ulogDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node7/data/tlog", "core":"testSimple1_shard2_replica_n4", "shared_storage":"true", "state":"active", "leader":"true"}, "core_node8":{ "dataDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node8/data/", "base_url":"https://127.0.0.1:46091/solr", "node_name":"127.0.0.1:46091_solr", "type":"NRT", "ulogDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node8/data/tlog", "core":"testSimple1_shard2_replica_n6", "shared_storage":"true", "state":"down"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"true", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:37853_solr, 127.0.0.1:44062_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/18)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node3":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node3/data/",
"base_url":"https://127.0.0.1:44062/solr",
"node_name":"127.0.0.1:44062_solr",
"type":"NRT",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node3/data/tlog",
"core":"testSimple1_shard1_replica_n1",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node5":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node5/data/",
"base_url":"https://127.0.0.1:44062/solr",
"node_name":"127.0.0.1:44062_solr",
"type":"NRT",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node5/data/tlog",
"core":"testSimple1_shard1_replica_n2",
"shared_storage":"true",
"state":"active"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node7":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node7/data/",
"base_url":"https://127.0.0.1:44062/solr",
"node_name":"127.0.0.1:44062_solr",
"type":"NRT",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node7/data/tlog",
"core":"testSimple1_shard2_replica_n4",
"shared_storage":"true",
"state":"active",
"leader":"true"},
"core_node8":{
"dataDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node8/data/",
"base_url":"https://127.0.0.1:46091/solr",
"node_name":"127.0.0.1:46091_solr",
"type":"NRT",
"ulogDir":"hdfs://lucene2-us-west.apache.org:38536/data/testSimple1/core_node8/data/tlog",
"core":"testSimple1_shard2_replica_n6",
"shared_storage":"true",
"state":"down"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"true",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([C094921560CA1E99:F827B6EB4739CA48]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([C094921560CA1E99:9DDC8C9CAF0CB8D6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([C094921560CA1E99:AD6836E8DA82E19E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:241)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12275 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster
[junit4] 2> 985260 INFO (SUITE-TestLargeCluster-seed#[C094921560CA1E99]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestLargeCluster_C094921560CA1E99-001/init-core-data-001
[junit4] 2> 985269 INFO (SUITE-TestLargeCluster-seed#[C094921560CA1E99]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 985270 INFO (SUITE-TestLargeCluster-seed#[C094921560CA1E99]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 985300 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 985301 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 985303 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10020_solr, 127.0.0.1:10023_solr, 127.0.0.1:10007_solr, 127.0.0.1:10015_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10005_solr, 127.0.0.1:10013_solr, 127.0.0.1:10021_solr, 127.0.0.1:10035_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10008_solr, 127.0.0.1:10019_solr, 127.0.0.1:10024_solr, 127.0.0.1:10027_solr, 127.0.0.1:10022_solr, 127.0.0.1:10039_solr, 127.0.0.1:10033_solr, 127.0.0.1:10025_solr, 127.0.0.1:10017_solr, 127.0.0.1:10002_solr, 127.0.0.1:10028_solr, 127.0.0.1:10036_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10006_solr, 127.0.0.1:10037_solr, 127.0.0.1:10009_solr, 127.0.0.1:10029_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10003_solr, 127.0.0.1:10011_solr]
[junit4] 2> 985303 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 985303 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 985305 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 985305 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 985305 DEBUG (ScheduledTrigger-1831-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 40
[junit4] 2> 985354 DEBUG (ScheduledTrigger-1831-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985355 INFO (TEST-TestLargeCluster.testBasic-seed#[C094921560CA1E99]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBasic
[junit4] 2> 985356 DEBUG (TEST-TestLargeCluster.testBasic-seed#[C094921560CA1E99]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 985356 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 985356 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 985356 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 985356 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 985375 INFO (TEST-TestLargeCluster.testBasic-seed#[C094921560CA1E99]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 985376 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 985376 DEBUG (TEST-TestLargeCluster.testBasic-seed#[C094921560CA1E99]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 985376 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 985377 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 985377 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 985377 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 985377 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 985377 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 985377 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 985377 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985397 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985418 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985438 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985458 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985479 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985483 DEBUG (simCloudManagerPool-1833-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 985483 DEBUG (simCloudManagerPool-1833-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 985483 DEBUG (simCloudManagerPool-1833-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 985483 DEBUG (simCloudManagerPool-1833-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 985484 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 985484 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 985484 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 985484 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985499 DEBUG (simCloudManagerPool-1833-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 985533 DEBUG (simCloudManagerPool-1833-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 985533 DEBUG (simCloudManagerPool-1833-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 985533 DEBUG (simCloudManagerPool-1833-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10082_solr, 127.0.0.1:10004_solr, 127.0.0.1:10018_solr, 127.0.0.1:10049_solr, 127.0.0.1:10020_solr, 127.0.0.1:10065_solr, 127.0.0.1:10021_solr, 127.0.0.1:10005_solr, 127.0.0.1:10066_solr, 127.0.0.1:10035_solr, 127.0.0.1:10051_solr, 127.0.0.1:10019_solr, 127.0.0.1:10096_solr, 127.0.0.1:10022_solr, 127.0.0.1:10033_solr, 127.0.0.1:10017_solr, 127.0.0.1:10047_solr, 127.0.0.1:10095_solr, 127.0.0.1:10081_solr, 127.0.0.1:10084_solr, 127.0.0.1:10036_solr, 127.0.0.1:10052_solr, 127.0.0.1:10006_solr, 127.0.0.1:10079_solr, 127.0.0.1:10098_solr, 127.0.0.1:10003_solr, 127.0.0.1:10001_solr, 127.0.0.1:10054_solr, 127.0.0.1:10099_solr, 127.0.0.1:10071_solr, 127.0.0.1:10023_solr, 127.0.0.1:10076_solr, 127.0.0.1:10007_solr, 127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10016_solr, 127.0.0.1:10070_solr, 127.0.0.1:10077_solr, 127.0.0.1:10038_solr, 127.0.0.1:10032_solr, 127.0.0.1:10093_solr, 127.0.0.1:10061_solr, 127.0.0.1:10067_solr, 127.0.0.1:10086_solr, 127.0.0.1:10089_solr, 127.0.0.1:10045_solr, 127.0.0.1:10042_solr, 127.0.0.1:10029_solr, 127.0.0.1:10083_solr, 127.0.0.1:10080_solr, 127.0.0.1:10048_solr, 127.0.0.1:10064_solr, 127.0.0.1:10043_solr, 127.0.0.1:10057_solr, 127.0.0.1:10074_solr, 127.0.0.1:10026_solr, 127.0.0.1:10012_solr, 127.0.0.1:10060_solr, 127.0.0.1:10073_solr, 127.0.0.1:10058_solr, 127.0.0.1:10013_solr, 127.0.0.1:10090_solr, 127.0.0.1:10088_solr, 127.0.0.1:10027_solr, 127.0.0.1:10039_solr, 127.0.0.1:10025_solr, 127.0.0.1:10044_solr, 127.0.0.1:10028_solr, 127.0.0.1:10092_solr, 127.0.0.1:10041_solr, 127.0.0.1:10087_solr, 127.0.0.1:10009_solr, 127.0.0.1:10000_solr, 127.0.0.1:10030_solr, 127.0.0.1:10014_solr, 127.0.0.1:10011_solr, 127.0.0.1:10040_solr, 127.0.0.1:10085_solr, 127.0.0.1:10046_solr, 127.0.0.1:10068_solr, 127.0.0.1:10063_solr, 127.0.0.1:10015_solr, 127.0.0.1:10069_solr, 127.0.0.1:10062_solr, 127.0.0.1:10008_solr, 127.0.0.1:10024_solr, 127.0.0.1:10050_solr, 127.0.0.1:10075_solr, 127.0.0.1:10078_solr, 127.0.0.1:10002_solr, 127.0.0.1:10097_solr, 127.0.0.1:10031_solr, 127.0.0.1:10034_solr, 127.0.0.1:10091_solr, 127.0.0.1:10094_solr, 127.0.0.1:10037_solr, 127.0.0.1:10053_solr, 127.0.0.1:10059_solr, 127.0.0.1:10056_solr, 127.0.0.1:10072_solr]
[junit4] 2> 985533 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 985533 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985533 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985533 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 985534 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 985554 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985561 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985574 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985597 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985597 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985617 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985618 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985638 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985638 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985658 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985658 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985686 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985686 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985707 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985707 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985727 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985727 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985747 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985747 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985767 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985768 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985792 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985792 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985828 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985828 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985848 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985849 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985869 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985869 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985905 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985905 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985926 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985926 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985962 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985962 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 985998 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 985998 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986027 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986027 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986047 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986047 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986067 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986068 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986100 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986100 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986120 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986120 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986156 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986156 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986193 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986193 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986237 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986237 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986270 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986270 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986315 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986316 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986351 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986351 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986371 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986371 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986403 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986404 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986436 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986436 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986456 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986456 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986481 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986481 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986501 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986501 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986533 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986533 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986553 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986553 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986598 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986598 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986634 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986634 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986812 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986812 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986833 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986833 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986853 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986853 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986873 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986874 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986898 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986898 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986918 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986918 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986938 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986938 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 986971 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 986971 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987003 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987003 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987040 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987040 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987072 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987072 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987092 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987092 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987113 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987113 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987141 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987141 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987173 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987174 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987210 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987210 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987238 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987238 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987258 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987258 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987283 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987283 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987360 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987360 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987396 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987396 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987433 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987433 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987469 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987469 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987542 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987542 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987556 DEBUG (simCloudManagerPool-1833-thread-79) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard2: {"core_node18":{
[junit4] 2> "node_name":"127.0.0.1:10029_solr",
[junit4] 2> "core":"testBasic_shard2_replica_n18",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 987556 DEBUG (simCloudManagerPool-1833-thread-81) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard4: {"core_node54":{
[junit4] 2> "node_name":"127.0.0.1:10065_solr",
[junit4] 2> "core":"testBasic_shard4_replica_t54",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"TLOG",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard4",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 987556 DEBUG (simCloudManagerPool-1833-thread-82) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard5: {"core_node65":{
[junit4] 2> "node_name":"127.0.0.1:10071_solr",
[junit4] 2> "core":"testBasic_shard5_replica_n65",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard5",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 987556 DEBUG (simCloudManagerPool-1833-thread-80) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard3: {"core_node39":{
[junit4] 2> "node_name":"127.0.0.1:10077_solr",
[junit4] 2> "core":"testBasic_shard3_replica_t39",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"TLOG",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 987556 DEBUG (simCloudManagerPool-1833-thread-83) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard1: {"core_node1":{
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2> "core":"testBasic_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 987571 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 100
[junit4] 2> 987571 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 100
[junit4] 2> 987573 INFO (TEST-TestLargeCluster.testBasic-seed#[C094921560CA1E99]) [ ] o.a.s.c.a.s.TestLargeCluster Ready after 2264ms
[junit4] 2> 987592 DEBUG (simCloudManagerPool-1833-thread-90) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard3: {"core_node32":{
[junit4] 2> "node_name":"127.0.0.1:10036_solr",
[junit4] 2> "core":"testBasic_shard3_replica_n32",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 95
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 95
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10019_solr
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10019_solr
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10036_solr
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10036_solr
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10007_solr
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10007_solr
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10077_solr
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10077_solr
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10013_solr
[junit4] 2> 987595 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10013_solr
[junit4] 2> 987594 INFO (simCloudManagerPool-1833-thread-100) [ ] o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 4883ms
[junit4] 2> 987655 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 95
[junit4] 2> 987655 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 95
[junit4] 2> 987656 INFO (simCloudManagerPool-1833-thread-105) [ ] o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 1779ms
[junit4] 2> 987692 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 94
[junit4] 2> 987692 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 94
[junit4] 2> 987692 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10017_solr
[junit4] 2> 987692 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10017_solr
[junit4] 2> 987728 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 94
[junit4] 2> 987728 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 94
[junit4] 2> 987765 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10013_solr, 127.0.0.1:10077_solr, 127.0.0.1:10019_solr, 127.0.0.1:10036_solr, 127.0.0.1:10007_solr]
[junit4] 2> 987765 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4cc5865d58765T8ypjpf6zgo6utxl7by97pbtxp",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":1350579944523621,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350579944523621,
[junit4] 2> 1350579942568571,
[junit4] 2> 1350579936692721,
[junit4] 2> 1350579938663521,
[junit4] 2> 1350579940618071],
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10013_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10007_solr"]}}
[junit4] 2> 987765 DEBUG (simCloudManagerPool-1833-thread-110) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for testBasic / shard3: {"core_node31":{
[junit4] 2> "node_name":"127.0.0.1:10054_solr",
[junit4] 2> "core":"testBasic_shard3_replica_n31",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic"}}
[junit4] 2> 987785 INFO (simCloudManagerPool-1833-thread-115) [ ] o.a.s.c.ActionThrottle Throttling leader attempts - waiting for 4001ms
[junit4] 2> 987858 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger]
[junit4] 2> 988009 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"4cc5865d58765T8ypjpf6zgo6utxl7by97pbtxp",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":1350579944523621,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350579944523621,
[junit4] 2> 1350579942568571,
[junit4] 2> 1350579936692721,
[junit4] 2> 1350579938663521,
[junit4] 2> 1350579940618071],
[junit4] 2> "_enqueue_time_":1350593123252721,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10013_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10007_solr"]}}
[junit4] 2> 988010 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"4cc5865d58765T8ypjpf6zgo6utxl7by97pbtxp",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":1350579944523621,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350579944523621,
[junit4] 2> 1350579942568571,
[junit4] 2> 1350579936692721,
[junit4] 2> 1350579938663521,
[junit4] 2> 1350579940618071],
[junit4] 2> "_enqueue_time_":1350593123252721,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10013_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10007_solr"]}} with context properties: {BEFORE_ACTION=[compute]}
[junit4] 2> 988158 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10072_solr&inPlaceMove=true&replica=core_node17
[junit4] 2> 988264 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10056_solr&inPlaceMove=true&replica=core_node39
[junit4] 2> 988321 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10059_solr&inPlaceMove=true&replica=core_node26
[junit4] 2> 988340 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10053_solr&inPlaceMove=true&replica=core_node32
[junit4] 2> 988373 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10037_solr&inPlaceMove=true&replica=core_node16
[junit4] 2> 988373 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 1350618828 sessionWrapper.createTime 1350600817828321, this.sessionWrapper.createTime 1350600817828321
[junit4] 2> 988374 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"4cc5865d58765T8ypjpf6zgo6utxl7by97pbtxp",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":1350579944523621,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350579944523621,
[junit4] 2> 1350579942568571,
[junit4] 2> 1350579936692721,
[junit4] 2> 1350579938663521,
[junit4] 2> 1350579940618071],
[junit4] 2> "_enqueue_time_":1350593123252721,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10013_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10007_solr"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@3e02bfde, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@5c186f94, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@6fe86fd6, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@73b038c5, org.apache.solr.client.solrj.request.CollectionAdminRequest$MoveReplica@58679522], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
[junit4] 2> 988374 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10072_solr&inPlaceMove=true&replica=core_node17
[junit4] 2> 988475 DEBUG (simCloudManagerPool-1833-thread-125) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node77":{
[junit4] 2> "core":"testBasic_shard2_replica_n76",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10072_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 988673 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10056_solr&inPlaceMove=true&replica=core_node39
[junit4] 2> 988789 DEBUG (simCloudManagerPool-1833-thread-133) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node79":{
[junit4] 2> "core":"testBasic_shard3_replica_t78",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10056_solr",
[junit4] 2> "type":"TLOG"}}
[junit4] 2> 988934 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10059_solr&inPlaceMove=true&replica=core_node26
[junit4] 2> 989034 DEBUG (simCloudManagerPool-1833-thread-141) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node81":{
[junit4] 2> "core":"testBasic_shard2_replica_p80",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10059_solr",
[junit4] 2> "type":"PULL"}}
[junit4] 2> 989146 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10053_solr&inPlaceMove=true&replica=core_node32
[junit4] 2> 989247 DEBUG (simCloudManagerPool-1833-thread-149) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node83":{
[junit4] 2> "core":"testBasic_shard3_replica_n82",
[junit4] 2> "shard":"shard3",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10053_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 989358 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10037_solr&inPlaceMove=true&replica=core_node16
[junit4] 2> 989459 DEBUG (simCloudManagerPool-1833-thread-157) [ ] o.a.s.c.a.s.SimClusterStateProvider -- new replica: {"core_node85":{
[junit4] 2> "core":"testBasic_shard2_replica_n84",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"testBasic",
[junit4] 2> "node_name":"127.0.0.1:10037_solr",
[junit4] 2> "type":"NRT"}}
[junit4] 2> 989572 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 989572 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
[junit4] 2> 989573 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 78199 ms for event id=4cc5865d58765T8ypjpf6zgo6utxl7by97pbtxp
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10055_solr
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10055_solr
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10010_solr
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10010_solr
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10013_solr, 127.0.0.1:10077_solr, 127.0.0.1:10017_solr, 127.0.0.1:10019_solr, 127.0.0.1:10036_solr, 127.0.0.1:10007_solr]
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10017_solr]
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4cc5865c6b0c3T8ypjpf6zgo6utxl7by97pbtxs",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":1350579943551171,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350579943551171,
[junit4] 2> 1350579941597221,
[junit4] 2> 1350584777947821,
[junit4] 2> 1350579935582171,
[junit4] 2> 1350579937662121,
[junit4] 2> 1350579939650871],
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10013_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10017_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10007_solr"]}}
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4cc5985fe82c5T8ypjpf6zgo6utxl7by97pbtxt",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":1350584779047621,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1350584779047621],
[junit4] 2> "nodeNames":["127.0.0.1:10017_solr"]}}
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger]
[junit4] 2> 989673 DEBUG (ScheduledTrigger-1834-thread-3) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger processor for lost nodes: [127.0.0.1:10017_solr] is not ready, will try later
[junit4] 2> 989674 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"4cc5865c6b0c3T8ypjpf6zgo6utxl7by97pbtxs",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":1350579943551171,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350579943551171,
[junit4] 2> 1350579941597221,
[junit4] 2> 1350584777947821,
[junit4] 2> 1350579935582171,
[junit4] 2> 1350579937662121,
[junit4] 2> 1350579939650871],
[junit4] 2> "_enqueue_time_":1350683832284671,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10013_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10017_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10007_solr"]}}
[junit4] 2> 989674 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"4cc5865c6b0c3T8ypjpf6zgo6utxl7by97pbtxs",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":1350579943551171,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350579943551171,
[junit4] 2> 1350579941597221,
[junit4] 2> 1350584777947821,
[junit4] 2> 1350579935582171,
[junit4] 2> 1350579937662121,
[junit4] 2> 1350579939650871],
[junit4] 2> "_enqueue_time_":1350683832284671,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10013_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10017_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10007_solr"]}} with context properties: {BEFORE_ACTION=[auto_add_replicas_plan]}
[junit4] 2> 989676 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 1350683958 sessionWrapper.createTime 1350683943725621, this.sessionWrapper.createTime 1350683943725621
[junit4] 2> 989676 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"4cc5865c6b0c3T8ypjpf6zgo6utxl7by97pbtxs",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":1350579943551171,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350579943551171,
[junit4] 2> 1350579941597221,
[junit4] 2> 1350584777947821,
[junit4] 2> 1350579935582171,
[junit4] 2> 1350579937662121,
[junit4] 2> 1350579939650871],
[junit4] 2> "_enqueue_time_":1350683832284671,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10013_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10017_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10007_solr"]}} with context properties: {AFTER_ACTION=[auto_add_replicas_plan], BEFORE_ACTION=[auto_add_replicas_plan, execute_plan]}
[junit4] 2> 989676 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ExecutePlanAction No operations to execute for event: {
[junit4] 2> "id":"4cc5865c6b0c3T8ypjpf6zgo6utxl7by97pbtxs",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":1350579943551171,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350579943551171,
[junit4] 2> 1350579941597221,
[junit4] 2> 1350584777947821,
[junit4] 2> 1350579935582171,
[junit4] 2> 1350579937662121,
[junit4] 2> 1350579939650871],
[junit4] 2> "_enqueue_time_":1350683832284671,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10013_solr",
[junit4] 2> "127.0.0.1:10077_solr",
[junit4] 2> "127.0.0.1:10017_solr",
[junit4] 2> "127.0.0.1:10019_solr",
[junit4] 2> "127.0.0.1:10036_solr",
[junit4] 2> "127.0.0.1:10007_solr"]}}
[junit4] 2> 989676 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 989676 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
[junit4] 2> 989676 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 128 ms for event id=4cc5865c6b0c3T8ypjpf6zgo6utxl7by97pbtxs
[junit4] 2> 989776 DEBUG (ScheduledTrigger-1834-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 92
[junit4] 2> 989776 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 92
[junit4] 2> 989776 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10055_solr, 127.0.0.1:10010_solr, 127.0.0.1:10017_solr]
[junit4] 2> 989776 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4cc70954e7fbbT8ypjpf6zgo6utxl7by97pbtxx",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":1350683820195771,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350683820195771,
[junit4] 2> 1350683822176021,
[junit4] 2> 1350584779047621],
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10055_solr",
[junit4] 2> "127.0.0.1:10010_solr",
[junit4] 2> "127.0.0.1:10017_solr"]}}
[junit4] 2> 989776 DEBUG (ScheduledTrigger-1834-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger]
[junit4] 2> 989777 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"4cc70954e7fbbT8ypjpf6zgo6utxl7by97pbtxx",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":1350683820195771,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350683820195771,
[junit4] 2> 1350683822176021,
[junit4] 2> 1350584779047621],
[junit4] 2> "_enqueue_time_":1350689004721471,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10055_solr",
[junit4] 2> "127.0.0.1:10010_solr",
[junit4] 2> "127.0.0.1:10017_solr"]}}
[junit4] 2> 989777 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"4cc70954e7fbbT8ypjpf6zgo6utxl7by97pbtxx",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":1350683820195771,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350683820195771,
[junit4] 2> 1350683822176021,
[junit4] 2> 1350584779047621],
[junit4] 2> "_enqueue_time_":1350689004721471,
[junit4] 2> "nodeNames":[
[junit4] 2> "127.0.0.1:10055_solr",
[junit4] 2> "127.0.0.1:10010_solr",
[junit4] 2> "127.0.0.1:10017_solr"]}} with context properties: {BEFORE_ACTION=[compute]}
[junit4] 2> 989801 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10094_solr&inPlaceMove=true&replica=core_node62
[junit4] 2> 989817 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10091_solr&inPlaceMove=true&replica=core_node71
[junit4] 2> 989819 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testBasic&targetNode=127.0.0.1:10034_solr&inPlaceMove=true&replica=core_node34
[junit4] 2> 989819 INFO (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 1350691150 sessionWrapper.createTime 1350689081474671, this.sessionWrapper.createTime 1350689081474671
[junit4] 2> 989819 DEBUG (AutoscalingActionExecutor-1835-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"4cc70954e7fbbT8ypjpf6zgo6utxl7by97pbtxx",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":1350683820195771,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[
[junit4] 2> 1350683820195771,
[junit4] 2> 1350683822176021,
[junit4] 2> 1350584779047621],
[junit4]
[...truncated too long message...]
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 51 /solr/collections/testSimple1/state.json
[junit4] 2> 22 /solr/collections/testSimple2/state.json
[junit4] 2> 12 /solr/collections/testSimple3/state.json
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/testSimple1/leader_elect/shard1/election/99411373393248265-core_node3-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/99411373393248265-127.0.0.1:44062_solr-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/99411373393248264-127.0.0.1:46091_solr-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/live_nodes
[junit4] 2> 6 /solr/collections
[junit4] 2>
[junit4] 2> 2206387 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C094921560CA1E99]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 2206419 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C094921560CA1E99]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 2206519 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C094921560CA1E99-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C094921560CA1E99-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:38536) [ ] o.a.h.h.s.d.IncrementalBlockReportManager IncrementalBlockReportManager interrupted
[junit4] 2> 2206520 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C094921560CA1E99-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C094921560CA1E99-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:38536) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1301848271-127.0.0.1-1516897177912 (Datanode Uuid 30d824ff-9ab6-46de-b0e3-cd1f8e44f8cf) service to lucene2-us-west.apache.org/127.0.0.1:38536
[junit4] 2> 2206521 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C094921560CA1E99]-worker) [ ] o.a.h.h.s.d.DirectoryScanner DirectoryScanner: shutdown has been called
[junit4] 2> 2206543 WARN (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C094921560CA1E99-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C094921560CA1E99-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:38536) [ ] o.a.h.h.s.d.DataNode Ending block pool service for: Block pool BP-1301848271-127.0.0.1-1516897177912 (Datanode Uuid a20035d1-c6e3-4dce-b516-a038e40a415d) service to lucene2-us-west.apache.org/127.0.0.1:38536
[junit4] 2> 2206594 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C094921560CA1E99]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0
[junit4] 2> 2206996 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C094921560CA1E99]-worker) [ ] o.m.log Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:0
[junit4] 2> 2207015 WARN (1745614664@qtp-2141866747-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:37880) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false. Rechecking.
[junit4] 2> 2207061 WARN (1745614664@qtp-2141866747-1 - Acceptor0 HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:37880) [ ] o.a.h.h.HttpServer2 HttpServer Acceptor: isRunning is false
[junit4] 2> 2207279 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[C094921560CA1E99]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35185 35185
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_C094921560CA1E99-001
[junit4] 2> Jan 25, 2018 4:22:32 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 34 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=428, maxMBSortInHeap=7.228904134347337, sim=RandomSimilarity(queryNorm=false): {}, locale=es-PR, timezone=Asia/Baghdad
[junit4] 2> NOTE: Linux 4.4.0-104-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=224755784,total=519045120
[junit4] 2> NOTE: All tests run in this JVM: [HighlighterConfigTest, RAMDirectoryFactoryTest, HdfsWriteToMultipleCollectionsTest, TestSolrCloudWithSecureImpersonation, TestSmileRequest, PreAnalyzedUpdateProcessorTest, CdcrBidirectionalTest, OverseerTaskQueueTest, SolrCoreCheckLockOnStartupTest, TestQuerySenderNoQuery, ZkNodePropsTest, XmlUpdateRequestHandlerTest, CollectionsAPIAsyncDistributedZkTest, TestCloudDeleteByQuery, UUIDFieldTest, TestIBSimilarityFactory, OverseerModifyCollectionTest, SolrIndexConfigTest, DocExpirationUpdateProcessorFactoryTest, TestPseudoReturnFields, TestRawResponseWriter, TestMaxScoreQueryParser, BasicDistributedZkTest, TestSkipOverseerOperations, SystemLogListenerTest, DistributedFacetPivotSmallAdvancedTest, TestTolerantUpdateProcessorRandomCloud, TestCSVResponseWriter, CloudExitableDirectoryReaderTest, TestSubQueryTransformerCrossCore, TestCloudPivotFacet, TestFastOutputStream, SolrMetricsIntegrationTest, TestConfig, TestStreamBody, TestTrieFacet, TestNoOpRegenerator, NodeAddedTriggerTest, ZkStateReaderTest, TestImpersonationWithHadoopAuth, TestDynamicLoading, HdfsRecoveryZkTest, TermVectorComponentDistributedTest, TestHighlightDedupGrouping, SolrIndexMetricsTest, RuleEngineTest, RemoteQueryErrorTest, TestSchemaSimilarityResource, TestLMDirichletSimilarityFactory, TestSort, TestUninvertingReader, TestHashPartitioner, SolrIndexSplitterTest, CustomCollectionTest, TestLeaderInitiatedRecoveryThread, LeaderInitiatedRecoveryOnShardRestartTest, SolrCloudExampleTest, UpdateLogTest, TestSolrXml, TestConfigReload, TestRandomFaceting, TestCloudSchemaless, HdfsDirectoryTest, AtomicUpdateProcessorFactoryTest, DistributedQueryComponentCustomSortTest, DirectUpdateHandlerTest, SpellingQueryConverterTest, ZkCLITest, OutputWriterTest, TestShardHandlerFactory, TestOnReconnectListenerSupport, CoreAdminRequestStatusTest, DefaultValueUpdateProcessorTest, TestComplexPhraseLeadingWildcard, StatsComponentTest, SuggestComponentTest, LeaderFailoverAfterPartitionTest, ShardRoutingTest, BlockCacheTest, ConfigSetsAPITest, TestSQLHandler, TestFieldTypeResource, TestConfigSetsAPIExclusivity, TestJoin, TestDistribIDF, UpdateParamsTest, ChaosMonkeySafeLeaderWithPullReplicasTest, TestSolrCLIRunExample, ChangedSchemaMergeTest, RequestHandlersTest, MissingSegmentRecoveryTest, TestExclusionRuleCollectionAccess, TestReplicationHandlerBackup, TestBinaryField, CustomHighlightComponentTest, TestMinMaxOnMultiValuedField, DistributedFacetSimpleRefinementLongTailTest, DistributedQueueTest, HdfsAutoAddReplicasIntegrationTest]
[junit4] Completed [340/767 (3!)] on J0 in 183.19s, 1 test, 1 failure <<< FAILURES!
[...truncated 37993 lines...]
-ecj-javadoc-lint-tests:
[mkdir] Created dir: /tmp/ecj1411638578
[ecj-lint] Compiling 19 source files to /tmp/ecj1411638578
[ecj-lint] ----------
[ecj-lint] 1. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/spatial3d/src/test/org/apache/lucene/spatial3d/geom/RandomPlaneTest.java (at line 20)
[ecj-lint] import java.util.ArrayList;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import java.util.ArrayList is never used
[ecj-lint] ----------
[ecj-lint] 2. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/spatial3d/src/test/org/apache/lucene/spatial3d/geom/RandomPlaneTest.java (at line 21)
[ecj-lint] import java.util.List;
[ecj-lint] ^^^^^^^^^^^^^^
[ecj-lint] The import java.util.List is never used
[ecj-lint] ----------
[ecj-lint] 2 problems (2 errors)
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:836: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/build.xml:101: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/build.xml:208: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2281: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2112: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-7.x/lucene/common-build.xml:2145: Compile failed; see the compiler error output for details.
Total time: 169 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any