You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/02/10 18:43:39 UTC
[JENKINS] Lucene-Solr-Tests-master - Build # 2306 - Still unstable
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2306/
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.ltr.feature.TestRankingFeature
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestRankingFeature: 1) Thread[id=476, name=qtp2104387739-476, state=TIMED_WAITING, group=TGRP-TestRankingFeature] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestRankingFeature:
1) Thread[id=476, name=qtp2104387739-476, state=TIMED_WAITING, group=TGRP-TestRankingFeature]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([4F0630586B6F6B25]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.ltr.feature.TestRankingFeature
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=476, name=qtp2104387739-476, state=TIMED_WAITING, group=TGRP-TestRankingFeature] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=476, name=qtp2104387739-476, state=TIMED_WAITING, group=TGRP-TestRankingFeature]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([4F0630586B6F6B25]:0)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([AB19DF3F7CBD5EE6:1115E8B0235588A9]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.junit.Assert.assertNull(Assert.java:562)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12945 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/init-core-data-001
[junit4] 2> 571379 WARN (SUITE-TriggerIntegrationTest-seed#[AB19DF3F7CBD5EE6]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 571380 INFO (SUITE-TriggerIntegrationTest-seed#[AB19DF3F7CBD5EE6]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 571381 INFO (SUITE-TriggerIntegrationTest-seed#[AB19DF3F7CBD5EE6]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 571381 INFO (SUITE-TriggerIntegrationTest-seed#[AB19DF3F7CBD5EE6]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 571381 INFO (SUITE-TriggerIntegrationTest-seed#[AB19DF3F7CBD5EE6]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001
[junit4] 2> 571382 INFO (SUITE-TriggerIntegrationTest-seed#[AB19DF3F7CBD5EE6]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 571385 INFO (Thread-1428) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 571385 INFO (Thread-1428) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 571386 ERROR (Thread-1428) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 571485 INFO (SUITE-TriggerIntegrationTest-seed#[AB19DF3F7CBD5EE6]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46429
[junit4] 2> 571488 INFO (zkConnectionManagerCallback-1081-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571497 INFO (jetty-launcher-1078-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 571497 INFO (jetty-launcher-1078-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 571503 INFO (jetty-launcher-1078-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 571503 INFO (jetty-launcher-1078-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 571504 INFO (jetty-launcher-1078-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 571505 INFO (jetty-launcher-1078-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2c2df94{/solr,null,AVAILABLE}
[junit4] 2> 571506 INFO (jetty-launcher-1078-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 571506 INFO (jetty-launcher-1078-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 571506 INFO (jetty-launcher-1078-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 571506 INFO (jetty-launcher-1078-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5f13ee72{/solr,null,AVAILABLE}
[junit4] 2> 571506 INFO (jetty-launcher-1078-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@31abdd27{SSL,[ssl, http/1.1]}{127.0.0.1:41486}
[junit4] 2> 571506 INFO (jetty-launcher-1078-thread-1) [ ] o.e.j.s.Server Started @573594ms
[junit4] 2> 571506 INFO (jetty-launcher-1078-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41486}
[junit4] 2> 571506 ERROR (jetty-launcher-1078-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 571506 INFO (jetty-launcher-1078-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@76911d83{SSL,[ssl, http/1.1]}{127.0.0.1:47389}
[junit4] 2> 571507 INFO (jetty-launcher-1078-thread-2) [ ] o.e.j.s.Server Started @573595ms
[junit4] 2> 571507 INFO (jetty-launcher-1078-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47389}
[junit4] 2> 571507 ERROR (jetty-launcher-1078-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 571507 INFO (jetty-launcher-1078-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 571507 INFO (jetty-launcher-1078-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 571507 INFO (jetty-launcher-1078-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 571507 INFO (jetty-launcher-1078-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T17:46:45.527Z
[junit4] 2> 571507 INFO (jetty-launcher-1078-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 571507 INFO (jetty-launcher-1078-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 571507 INFO (jetty-launcher-1078-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 571507 INFO (jetty-launcher-1078-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T17:46:45.527Z
[junit4] 2> 571514 INFO (zkConnectionManagerCallback-1083-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571514 INFO (zkConnectionManagerCallback-1085-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571515 INFO (jetty-launcher-1078-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 571515 INFO (jetty-launcher-1078-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 571524 INFO (jetty-launcher-1078-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46429/solr
[junit4] 2> 571525 INFO (jetty-launcher-1078-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46429/solr
[junit4] 2> 571526 INFO (zkConnectionManagerCallback-1089-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571528 INFO (zkConnectionManagerCallback-1093-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571528 INFO (zkConnectionManagerCallback-1095-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571530 INFO (zkConnectionManagerCallback-1097-thread-1-processing-n:127.0.0.1:47389_solr) [n:127.0.0.1:47389_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571563 INFO (jetty-launcher-1078-thread-1) [n:127.0.0.1:41486_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 571564 INFO (jetty-launcher-1078-thread-1) [n:127.0.0.1:41486_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41486_solr
[junit4] 2> 571565 INFO (jetty-launcher-1078-thread-1) [n:127.0.0.1:41486_solr ] o.a.s.c.Overseer Overseer (id=73272416262160389-127.0.0.1:41486_solr-n_0000000000) starting
[junit4] 2> 571573 INFO (jetty-launcher-1078-thread-1) [n:127.0.0.1:41486_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41486_solr
[junit4] 2> 571577 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 571578 INFO (OverseerStateUpdate-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 571586 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=2060221275
[junit4] 2> 571586 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 571588 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:41486_solr]
[junit4] 2> 571588 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 571588 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 571586 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=1960112641
[junit4] 2> 571589 ERROR (Finalizer) [ ] o.a.s.c.CoreContainer CoreContainer was not close prior to finalize(), indicates a bug -- POSSIBLE RESOURCE LEAK!!! instance=2140130134
[junit4] 2> 571591 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 571591 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 571591 DEBUG (ScheduledTrigger-2068-thread-1) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 571624 INFO (jetty-launcher-1078-thread-2) [n:127.0.0.1:47389_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 571626 INFO (jetty-launcher-1078-thread-2) [n:127.0.0.1:47389_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 571628 INFO (jetty-launcher-1078-thread-2) [n:127.0.0.1:47389_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47389_solr
[junit4] 2> 571629 INFO (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 571630 INFO (zkCallback-1096-thread-1-processing-n:127.0.0.1:47389_solr) [n:127.0.0.1:47389_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 571640 INFO (jetty-launcher-1078-thread-1) [n:127.0.0.1:41486_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41486.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 571646 INFO (jetty-launcher-1078-thread-1) [n:127.0.0.1:41486_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41486.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 571647 INFO (jetty-launcher-1078-thread-1) [n:127.0.0.1:41486_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41486.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 571648 INFO (jetty-launcher-1078-thread-1) [n:127.0.0.1:41486_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001/node2/.
[junit4] 2> 571670 INFO (jetty-launcher-1078-thread-2) [n:127.0.0.1:47389_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47389.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 571677 INFO (jetty-launcher-1078-thread-2) [n:127.0.0.1:47389_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47389.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 571678 INFO (jetty-launcher-1078-thread-2) [n:127.0.0.1:47389_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47389.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 571679 INFO (jetty-launcher-1078-thread-2) [n:127.0.0.1:47389_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001/node1/.
[junit4] 2> 571699 INFO (zkConnectionManagerCallback-1103-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571703 INFO (zkConnectionManagerCallback-1107-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571705 INFO (SUITE-TriggerIntegrationTest-seed#[AB19DF3F7CBD5EE6]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 571706 INFO (SUITE-TriggerIntegrationTest-seed#[AB19DF3F7CBD5EE6]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46429/solr ready
[junit4] 2> 571725 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
[junit4] 2> 571725 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 571726 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 571726 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 571732 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 571733 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 571733 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 571758 INFO (qtp1225437743-5805) [n:127.0.0.1:41486_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 571783 INFO (qtp1009117156-5816) [n:127.0.0.1:47389_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 571784 DEBUG (qtp1225437743-5807) [n:127.0.0.1:41486_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 571786 INFO (qtp1225437743-5807) [n:127.0.0.1:41486_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=31
[junit4] 2> 571786 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 571790 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:41486_solr, 127.0.0.1:47389_solr]
[junit4] 2> 571790 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 571790 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 571791 INFO (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 571793 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 571793 DEBUG (ScheduledTrigger-2068-thread-1) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 571800 DEBUG (ScheduledTrigger-2068-thread-1) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 571800 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 571801 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 571802 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 571802 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 571802 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 571802 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@dbc4866{/solr,null,AVAILABLE}
[junit4] 2> 571803 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@78116463{SSL,[ssl, http/1.1]}{127.0.0.1:35355}
[junit4] 2> 571803 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.Server Started @573891ms
[junit4] 2> 571803 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35355}
[junit4] 2> 571803 ERROR (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 571804 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 571804 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 571804 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 571804 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T17:46:45.824Z
[junit4] 2> 571806 INFO (zkConnectionManagerCallback-1109-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571806 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 571814 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46429/solr
[junit4] 2> 571815 INFO (zkConnectionManagerCallback-1113-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571818 INFO (zkConnectionManagerCallback-1115-thread-1-processing-n:127.0.0.1:35355_solr) [n:127.0.0.1:35355_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 571824 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:35355_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 571826 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:35355_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 571829 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:35355_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35355_solr
[junit4] 2> 571830 INFO (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 571830 INFO (zkCallback-1096-thread-1-processing-n:127.0.0.1:47389_solr) [n:127.0.0.1:47389_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 571831 INFO (zkCallback-1114-thread-1-processing-n:127.0.0.1:35355_solr) [n:127.0.0.1:35355_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 571831 INFO (zkCallback-1106-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 571903 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:35355_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35355.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 571912 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:35355_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35355.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 571912 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:35355_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35355.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 571913 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:35355_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001/node3/.
[junit4] 2> 572802 DEBUG (ScheduledTrigger-2068-thread-2) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 572802 DEBUG (ScheduledTrigger-2068-thread-2) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 572802 DEBUG (ScheduledTrigger-2068-thread-2) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:35355_solr at time 18536717485882624
[junit4] 2> 573804 DEBUG (ScheduledTrigger-2068-thread-2) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 573804 DEBUG (ScheduledTrigger-2068-thread-2) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 574804 DEBUG (ScheduledTrigger-2068-thread-2) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 574804 DEBUG (ScheduledTrigger-2068-thread-2) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 574804 DEBUG (ScheduledTrigger-2068-thread-2) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:35355_solr] added at times [18536717485882624], now=18536719487756706
[junit4] 2> 574806 DEBUG (ScheduledTrigger-2068-thread-2) [n:127.0.0.1:41486_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"41db0bdd0c3d00Tc65cxi7kw1n3szpc135m6l9g3",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":18536717485882624,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18536717485882624],
[junit4] 2> "nodeNames":["127.0.0.1:35355_solr"]}}
[junit4] 2> 574806 DEBUG (ScheduledTrigger-2068-thread-2) [n:127.0.0.1:41486_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger]
[junit4] 2> 574808 INFO (ScheduledTrigger-2068-thread-2) [n:127.0.0.1:41486_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 574813 DEBUG (AutoscalingActionExecutor-2069-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"41db0bdd0c3d00Tc65cxi7kw1n3szpc135m6l9g3",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":18536717485882624,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18536717485882624],
[junit4] 2> "_enqueue_time_":18536719490199994,
[junit4] 2> "nodeNames":["127.0.0.1:35355_solr"]}}
[junit4] 2> 574819 DEBUG (AutoscalingActionExecutor-2069-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 5000ms
[junit4] 2> 574819 DEBUG (AutoscalingActionExecutor-2069-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 5 ms for event id=41db0bdd0c3d00Tc65cxi7kw1n3szpc135m6l9g3
[junit4] 2> 574837 INFO (qtp998370872-5865) [n:127.0.0.1:35355_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 574839 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 574841 INFO (qtp1009117156-5818) [n:127.0.0.1:47389_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 574842 DEBUG (qtp998370872-5871) [n:127.0.0.1:35355_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 574842 INFO (qtp998370872-5871) [n:127.0.0.1:35355_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=8
[junit4] 2> 574842 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 574843 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:41486_solr, 127.0.0.1:35355_solr, 127.0.0.1:47389_solr]
[junit4] 2> 574843 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 574843 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 574843 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 574843 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 576843 INFO (TEST-TriggerIntegrationTest.testNodeAddedTrigger-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTrigger
[junit4] 2> 576860 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTriggerRestoreState
[junit4] 2> 576861 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 5
[junit4] 2> 576861 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 576861 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 576862 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 576862 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 576863 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 576864 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 576865 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 576865 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 576865 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 576866 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3672ad33{/solr,null,AVAILABLE}
[junit4] 2> 576866 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2377f4ee{SSL,[ssl, http/1.1]}{127.0.0.1:44100}
[junit4] 2> 576866 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.Server Started @578954ms
[junit4] 2> 576866 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44100}
[junit4] 2> 576866 ERROR (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 576866 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 576867 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 576867 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 576867 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T17:46:50.887Z
[junit4] 2> 576868 INFO (zkConnectionManagerCallback-1119-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 576869 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 576877 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46429/solr
[junit4] 2> 576878 INFO (zkConnectionManagerCallback-1123-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 576880 INFO (zkConnectionManagerCallback-1125-thread-1-processing-n:127.0.0.1:44100_solr) [n:127.0.0.1:44100_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 576885 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:44100_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 576887 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:44100_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 576889 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:44100_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44100_solr
[junit4] 2> 576890 INFO (zkCallback-1106-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 576890 INFO (zkCallback-1096-thread-1-processing-n:127.0.0.1:47389_solr) [n:127.0.0.1:47389_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 576890 INFO (zkCallback-1114-thread-1-processing-n:127.0.0.1:35355_solr) [n:127.0.0.1:35355_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 576890 INFO (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 576899 INFO (zkCallback-1124-thread-1-processing-n:127.0.0.1:44100_solr) [n:127.0.0.1:44100_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 576961 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:44100_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44100.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 576971 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:44100_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44100.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 576971 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:44100_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44100.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 576972 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:44100_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001/node4/.
[junit4] 2> 577016 INFO (qtp998370872-5872) [n:127.0.0.1:35355_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 577019 INFO (qtp1225437743-5807) [n:127.0.0.1:41486_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 577041 INFO (qtp377714589-5896) [n:127.0.0.1:44100_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 577044 INFO (qtp1009117156-5816) [n:127.0.0.1:47389_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 577045 DEBUG (qtp998370872-5869) [n:127.0.0.1:35355_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 577045 INFO (qtp998370872-5869) [n:127.0.0.1:35355_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=32
[junit4] 2> 577046 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 577046 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:41486_solr, 127.0.0.1:35355_solr, 127.0.0.1:47389_solr, 127.0.0.1:44100_solr]
[junit4] 2> 577046 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 577046 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@2377f4ee{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 577047 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=551745611
[junit4] 2> 577047 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 577047 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@694d5e82: rootName = solr_44100, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@723bfd5a
[junit4] 2> 577047 INFO (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 577047 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 577048 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 577053 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 577053 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4228226: rootName = solr_44100, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@3bdb6685
[junit4] 2> 577053 DEBUG (ScheduledTrigger-2068-thread-3) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 4
[junit4] 2> 577057 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 577057 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4c1eb4af: rootName = solr_44100, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@24fed25a
[junit4] 2> 577058 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44100_solr
[junit4] 2> 577059 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 577059 INFO (zkCallback-1106-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 577059 INFO (zkCallback-1114-thread-1-processing-n:127.0.0.1:35355_solr) [n:127.0.0.1:35355_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 577059 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 577059 INFO (zkCallback-1096-thread-1-processing-n:127.0.0.1:47389_solr) [n:127.0.0.1:47389_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 577059 INFO (zkCallback-1124-thread-1-processing-n:127.0.0.1:44100_solr) [n:127.0.0.1:44100_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 577060 INFO (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 577061 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3672ad33{/solr,null,UNAVAILABLE}
[junit4] 2> 577061 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 578054 DEBUG (ScheduledTrigger-2068-thread-3) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 3
[junit4] 2> 578054 DEBUG (ScheduledTrigger-2068-thread-3) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:44100_solr
[junit4] 2> 578567 INFO (qtp998370872-5871) [n:127.0.0.1:35355_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 578570 INFO (qtp1225437743-5808) [n:127.0.0.1:41486_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 578574 INFO (qtp1009117156-5814) [n:127.0.0.1:47389_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 578574 DEBUG (qtp1225437743-5804) [n:127.0.0.1:41486_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 578574 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=10
[junit4] 2> 578575 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 578575 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:41486_solr, 127.0.0.1:35355_solr, 127.0.0.1:47389_solr]
[junit4] 2> 578575 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 578576 INFO (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 578576 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:44100_solr
[junit4] 2> 578576 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 578576 DEBUG (ScheduledTrigger-2068-thread-4) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 3
[junit4] 2> 578576 DEBUG (ScheduledTrigger-2068-thread-4) [n:127.0.0.1:41486_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:44100_solr]
[junit4] 2> 578577 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 578577 DEBUG (ScheduledTrigger-2068-thread-4) [n:127.0.0.1:41486_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"41db0d352e6d28Tc65cxi7kw1n3szpc135m6l9g5",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":18536723259485480,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18536723259485480],
[junit4] 2> "nodeNames":["127.0.0.1:44100_solr"]}}
[junit4] 2> 578577 DEBUG (ScheduledTrigger-2068-thread-4) [n:127.0.0.1:41486_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_restore_trigger]
[junit4] 2> 578577 INFO (ScheduledTrigger-2068-thread-4) [n:127.0.0.1:41486_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 578578 DEBUG (AutoscalingActionExecutor-2069-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"41db0d352e6d28Tc65cxi7kw1n3szpc135m6l9g5",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":18536723259485480,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18536723259485480],
[junit4] 2> "_enqueue_time_":18536723260332374,
[junit4] 2> "nodeNames":["127.0.0.1:44100_solr"]}}
[junit4] 2> 578578 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTriggerRestoreState
[junit4] 2> 578579 DEBUG (AutoscalingActionExecutor-2069-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_restore_trigger after 5000ms
[junit4] 2> 578579 DEBUG (AutoscalingActionExecutor-2069-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=41db0d352e6d28Tc65cxi7kw1n3szpc135m6l9g5
[junit4] 2> 578602 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
[junit4] 2> 578602 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 8
[junit4] 2> 578602 DEBUG (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 578602 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 578603 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 578603 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 578604 DEBUG (OverseerAutoScalingTriggerThread-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 578607 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 578608 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 578608 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 578608 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 578608 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6c1b6ac0{/solr,null,AVAILABLE}
[junit4] 2> 578609 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@46f76d2f{SSL,[ssl, http/1.1]}{127.0.0.1:60625}
[junit4] 2> 578609 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.Server Started @580698ms
[junit4] 2> 578610 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60625}
[junit4] 2> 578610 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 578610 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 578610 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 578610 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 578610 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T17:46:52.630Z
[junit4] 2> 578612 INFO (zkConnectionManagerCallback-1129-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578612 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 578619 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46429/solr
[junit4] 2> 578619 INFO (zkConnectionManagerCallback-1133-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578621 INFO (zkConnectionManagerCallback-1135-thread-1-processing-n:127.0.0.1:60625_solr) [n:127.0.0.1:60625_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578631 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:60625_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 578632 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:60625_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 578634 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:60625_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60625_solr
[junit4] 2> 578635 INFO (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 578635 INFO (zkCallback-1114-thread-1-processing-n:127.0.0.1:35355_solr) [n:127.0.0.1:35355_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 578635 INFO (zkCallback-1106-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 578635 INFO (zkCallback-1096-thread-1-processing-n:127.0.0.1:47389_solr) [n:127.0.0.1:47389_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 578636 INFO (zkCallback-1134-thread-1-processing-n:127.0.0.1:60625_solr) [n:127.0.0.1:60625_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 578694 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:60625_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60625.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 578703 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:60625_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60625.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 578703 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:60625_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60625.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 578704 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:60625_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001/node5/.
[junit4] 2> 578745 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 578746 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 578746 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 578746 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 578746 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4878e07f{/solr,null,AVAILABLE}
[junit4] 2> 578747 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7fea67d{SSL,[ssl, http/1.1]}{127.0.0.1:58228}
[junit4] 2> 578747 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.Server Started @580835ms
[junit4] 2> 578747 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58228}
[junit4] 2> 578747 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 578747 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 578747 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 578747 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 578747 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T17:46:52.767Z
[junit4] 2> 578749 INFO (zkConnectionManagerCallback-1139-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578750 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 578756 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46429/solr
[junit4] 2> 578757 INFO (zkConnectionManagerCallback-1143-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578760 INFO (zkConnectionManagerCallback-1145-thread-1-processing-n:127.0.0.1:58228_solr) [n:127.0.0.1:58228_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578764 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:58228_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 578765 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:58228_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 578767 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:58228_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58228_solr
[junit4] 2> 578768 INFO (zkCallback-1096-thread-1-processing-n:127.0.0.1:47389_solr) [n:127.0.0.1:47389_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 578768 INFO (zkCallback-1134-thread-1-processing-n:127.0.0.1:60625_solr) [n:127.0.0.1:60625_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 578768 INFO (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 578768 INFO (zkCallback-1106-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 578768 INFO (zkCallback-1114-thread-1-processing-n:127.0.0.1:35355_solr) [n:127.0.0.1:35355_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 578769 INFO (zkCallback-1144-thread-1-processing-n:127.0.0.1:58228_solr) [n:127.0.0.1:58228_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 578822 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:58228_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58228.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 578829 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:58228_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58228.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 578830 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:58228_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58228.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 578831 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:58228_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001/node6/.
[junit4] 2> 578895 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 578896 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 578896 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 578896 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 578897 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@66474db6{/solr,null,AVAILABLE}
[junit4] 2> 578897 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4863a987{SSL,[ssl, http/1.1]}{127.0.0.1:36500}
[junit4] 2> 578897 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.e.j.s.Server Started @580985ms
[junit4] 2> 578897 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36500}
[junit4] 2> 578897 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 578897 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 578897 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 578897 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 578898 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T17:46:52.918Z
[junit4] 2> 578899 INFO (zkConnectionManagerCallback-1149-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578899 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 578906 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46429/solr
[junit4] 2> 578913 INFO (zkConnectionManagerCallback-1153-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578916 INFO (zkConnectionManagerCallback-1155-thread-1-processing-n:127.0.0.1:36500_solr) [n:127.0.0.1:36500_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578920 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:36500_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 578921 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:36500_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 578923 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:36500_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36500_solr
[junit4] 2> 578924 INFO (zkCallback-1096-thread-1-processing-n:127.0.0.1:47389_solr) [n:127.0.0.1:47389_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 578924 INFO (zkCallback-1106-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 578924 INFO (zkCallback-1144-thread-1-processing-n:127.0.0.1:58228_solr) [n:127.0.0.1:58228_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 578924 INFO (zkCallback-1134-thread-1-processing-n:127.0.0.1:60625_solr) [n:127.0.0.1:60625_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 578924 INFO (zkCallback-1114-thread-1-processing-n:127.0.0.1:35355_solr) [n:127.0.0.1:35355_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 578925 INFO (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 578929 INFO (zkCallback-1154-thread-1-processing-n:127.0.0.1:36500_solr) [n:127.0.0.1:36500_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 578996 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:36500_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36500.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 579005 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:36500_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36500.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 579005 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:36500_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36500.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 579006 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[AB19DF3F7CBD5EE6]) [n:127.0.0.1:36500_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001/node7/.
[junit4] 2> 579065 INFO (qtp998370872-5869) [n:127.0.0.1:35355_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=collection1&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 579067 INFO (OverseerThreadFactory-2070-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 579181 INFO (OverseerStateUpdate-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:58228/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 579182 INFO (OverseerStateUpdate-73272416262160389-127.0.0.1:41486_solr-n_0000000000) [n:127.0.0.1:41486_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:41486/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 579386 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n2&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 579386 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 579400 INFO (qtp1819489379-5947) [n:127.0.0.1:58228_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 579401 INFO (qtp1819489379-5947) [n:127.0.0.1:58228_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 579503 INFO (zkCallback-1144-thread-1-processing-n:127.0.0.1:58228_solr) [n:127.0.0.1:58228_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [6])
[junit4] 2> 579503 INFO (zkCallback-1094-thread-1-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [6])
[junit4] 2> 579504 INFO (zkCallback-1094-thread-2-processing-n:127.0.0.1:41486_solr) [n:127.0.0.1:41486_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [6])
[junit4] 2> 579504 INFO (zkCallback-1144-thread-2-processing-n:127.0.0.1:58228_solr) [n:127.0.0.1:58228_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [6])
[junit4] 2> 580399 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 580406 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema [collection1_shard1_replica_n2] Schema name=minimal
[junit4] 2> 580408 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 580408 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n2' using configuration from collection collection1, trusted=true
[junit4] 2> 580409 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41486.solr.core.collection1.shard1.replica_n2' (registry 'solr.core.collection1.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 580409 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 580409 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore [[collection1_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001/node2/collection1_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001/node2/./collection1_shard1_replica_n2/data/]
[junit4] 2> 580412 INFO (qtp1819489379-5947) [n:127.0.0.1:58228_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 580418 INFO (qtp1819489379-5947) [n:127.0.0.1:58228_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=minimal
[junit4] 2> 580420 INFO (qtp1819489379-5947) [n:127.0.0.1:58228_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 580420 INFO (qtp1819489379-5947) [n:127.0.0.1:58228_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from collection collection1, trusted=true
[junit4] 2> 580420 INFO (qtp1819489379-5947) [n:127.0.0.1:58228_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58228.solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@23e19f4f
[junit4] 2> 580421 INFO (qtp1819489379-5947) [n:127.0.0.1:58228_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 580421 INFO (qtp1819489379-5947) [n:127.0.0.1:58228_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001/node6/collection1_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_AB19DF3F7CBD5EE6-001/tempDir-001/node6/./collection1_shard1_replica_n1/data/]
[junit4] 2> 580473 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 580473 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 580474 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 580474 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 580475 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@12d8130b[collection1_shard1_replica_n2] main]
[junit4] 2> 580476 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 580477 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 580477 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 580478 INFO (qtp1819489379-5947) [n:127.0.0.1:58228_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 580478 INFO (qtp1819489379-5947) [n:127.0.0.1:58228_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 580478 INFO (searcherExecutor-2106-thread-1-processing-n:127.0.0.1:41486_solr x:collection1_shard1_replica_n2 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore [collection1_shard1_replica_n2] Registered new searcher Searcher@12d8130b[collection1_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 580478 INFO (qtp1225437743-5804) [n:127.0.0.1:41486_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592037017647054848
[junit4] 2> 580479 INFO (qtp1819489379
[...truncated too long message...]
tRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@13f5d4f5{/solr,null,UNAVAILABLE}
[junit4] 2> 28776 INFO (SUITE-TestRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 58774 WARN (SUITE-TestRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.e.j.u.t.QueuedThreadPool QueuedThreadPool@qtp2104387739{STOPPING,8<=8<=10000,i=0,q=1} Couldn't stop Thread[qtp2104387739-476,5,TGRP-TestRankingFeature]
[junit4] 2> 58777 INFO (SUITE-TestRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 58777 INFO (SUITE-TestRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=217076634
[junit4] 2> 58777 INFO (SUITE-TestRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 58777 INFO (SUITE-TestRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@fe7ac2e: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / null
[junit4] 2> 58778 INFO (SUITE-TestRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 58778 INFO (SUITE-TestRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@22a2cdd5: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / null
[junit4] 2> 58778 INFO (SUITE-TestRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 58778 INFO (SUITE-TestRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f0737e1: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / null
[junit4] 2> 58779 INFO (coreCloseExecutor-382-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@4e68a4bd
[junit4] 2> 58779 INFO (coreCloseExecutor-382-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=1315480765
[junit4] 2> 58779 INFO (coreCloseExecutor-382-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2aa733e9: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / null
[junit4] 2> 58782 INFO (SUITE-TestRankingFeature-seed#[4F0630586B6F6B25]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 10, 2018 6:34:30 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 10, 2018 6:34:50 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestRankingFeature:
[junit4] 2> 1) Thread[id=476, name=qtp2104387739-476, state=TIMED_WAITING, group=TGRP-TestRankingFeature]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 10, 2018 6:34:50 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=476, name=qtp2104387739-476, state=TIMED_WAITING, group=TGRP-TestRankingFeature]
[junit4] 2> Feb 10, 2018 6:34:53 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=476, name=qtp2104387739-476, state=TIMED_WAITING, group=TGRP-TestRankingFeature]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {_version_=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), popularity=Lucene50(blocksize=128), description=PostingsFormat(name=LuceneVarGapFixedInterval), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), text=Lucene50(blocksize=128), title=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{}, maxPointsInLeafNode=1467, maxMBSortInHeap=6.398991991189087, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7cab733c), locale=sq-AL, timezone=Asia/Hebron
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=273587056,total=339214336
[junit4] 2> NOTE: All tests run in this JVM: [TestLTRQParserExplain, TestValueFeature, TestMultipleAdditiveTreesModel, TestModelManagerPersistence, TestEdisMaxSolrFeature, TestRerankBase, TestExternalValueFeatures, TestFeatureExtractionFromMultipleSegments, TestFieldLengthFeature, TestFieldValueFeature, TestNoMatchSolrFeature, TestOriginalScoreFeature, TestOriginalScoreScorer, TestRankingFeature]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestRankingFeature -Dtests.seed=4F0630586B6F6B25 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=sq-AL -Dtests.timezone=Asia/Hebron -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestRankingFeature (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestRankingFeature:
[junit4] > 1) Thread[id=476, name=qtp2104387739-476, state=TIMED_WAITING, group=TGRP-TestRankingFeature]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([4F0630586B6F6B25]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=476, name=qtp2104387739-476, state=TIMED_WAITING, group=TGRP-TestRankingFeature]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([4F0630586B6F6B25]:0)
[junit4] Completed [35/35 (1!)] on J1 in 54.09s, 1 test, 2 errors <<< FAILURES!
[...truncated 40694 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2315 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2315/
1 tests failed.
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([F0965BF548E1B463:9D6AFF08F2A94B64]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:274)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12673 lines...]
[junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/init-core-data-001
[junit4] 2> 1446913 WARN (SUITE-AutoscalingHistoryHandlerTest-seed#[F0965BF548E1B463]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=14 numCloses=14
[junit4] 2> 1446925 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[F0965BF548E1B463]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1446939 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[F0965BF548E1B463]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1446939 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[F0965BF548E1B463]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1446940 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[F0965BF548E1B463]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001
[junit4] 2> 1446940 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[F0965BF548E1B463]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1446959 INFO (Thread-1564) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1446959 INFO (Thread-1564) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1446975 ERROR (Thread-1564) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1447059 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[F0965BF548E1B463]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46513
[junit4] 2> 1447083 INFO (zkConnectionManagerCallback-985-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1447139 INFO (jetty-launcher-982-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1447140 INFO (jetty-launcher-982-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1447141 INFO (jetty-launcher-982-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1447141 INFO (jetty-launcher-982-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1447141 INFO (jetty-launcher-982-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1447141 INFO (jetty-launcher-982-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1447142 INFO (jetty-launcher-982-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2c3d9faf{/solr,null,AVAILABLE}
[junit4] 2> 1447143 INFO (jetty-launcher-982-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@31a1b0a3{HTTP/1.1,[http/1.1]}{127.0.0.1:45875}
[junit4] 2> 1447143 INFO (jetty-launcher-982-thread-1) [ ] o.e.j.s.Server Started @1458308ms
[junit4] 2> 1447143 INFO (jetty-launcher-982-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45875}
[junit4] 2> 1447143 ERROR (jetty-launcher-982-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1447143 INFO (jetty-launcher-982-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1447143 INFO (jetty-launcher-982-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1447143 INFO (jetty-launcher-982-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1447143 INFO (jetty-launcher-982-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T08:49:11.059Z
[junit4] 2> 1447145 INFO (jetty-launcher-982-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1447145 INFO (jetty-launcher-982-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1447145 INFO (jetty-launcher-982-thread-3) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1447145 INFO (jetty-launcher-982-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1225e503{/solr,null,AVAILABLE}
[junit4] 2> 1447145 INFO (jetty-launcher-982-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4d166e96{HTTP/1.1,[http/1.1]}{127.0.0.1:42339}
[junit4] 2> 1447145 INFO (jetty-launcher-982-thread-3) [ ] o.e.j.s.Server Started @1458310ms
[junit4] 2> 1447145 INFO (jetty-launcher-982-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42339}
[junit4] 2> 1447145 ERROR (jetty-launcher-982-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1447145 INFO (jetty-launcher-982-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1447146 INFO (jetty-launcher-982-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1447146 INFO (jetty-launcher-982-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1447146 INFO (jetty-launcher-982-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T08:49:11.062Z
[junit4] 2> 1447191 INFO (jetty-launcher-982-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1447191 INFO (jetty-launcher-982-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1447191 INFO (jetty-launcher-982-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1447197 INFO (jetty-launcher-982-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@55e00862{/solr,null,AVAILABLE}
[junit4] 2> 1447198 INFO (jetty-launcher-982-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@53d427a5{HTTP/1.1,[http/1.1]}{127.0.0.1:46559}
[junit4] 2> 1447198 INFO (jetty-launcher-982-thread-2) [ ] o.e.j.s.Server Started @1458363ms
[junit4] 2> 1447198 INFO (jetty-launcher-982-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46559}
[junit4] 2> 1447198 ERROR (jetty-launcher-982-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1447198 INFO (jetty-launcher-982-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1447198 INFO (jetty-launcher-982-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1447198 INFO (jetty-launcher-982-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1447198 INFO (jetty-launcher-982-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-12T08:49:11.114Z
[junit4] 2> 1447370 INFO (zkConnectionManagerCallback-987-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1447576 INFO (jetty-launcher-982-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1447610 INFO (zkConnectionManagerCallback-989-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1447614 INFO (zkConnectionManagerCallback-991-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1447678 INFO (jetty-launcher-982-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1447678 INFO (jetty-launcher-982-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1447757 INFO (jetty-launcher-982-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46513/solr
[junit4] 2> 1447794 INFO (jetty-launcher-982-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46513/solr
[junit4] 2> 1447915 INFO (jetty-launcher-982-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46513/solr
[junit4] 2> 1447960 INFO (zkConnectionManagerCallback-995-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1448029 INFO (zkConnectionManagerCallback-999-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1448066 INFO (zkConnectionManagerCallback-1003-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1448078 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1002d76ab450004, likely client has closed socket
[junit4] 2> 1448140 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1002d76ab450006, likely client has closed socket
[junit4] 2> 1448191 INFO (zkConnectionManagerCallback-1005-thread-1-processing-n:127.0.0.1:42339_solr) [n:127.0.0.1:42339_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1448246 INFO (zkConnectionManagerCallback-1007-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1448285 INFO (zkConnectionManagerCallback-1009-thread-1-processing-n:127.0.0.1:46559_solr) [n:127.0.0.1:46559_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1448654 INFO (jetty-launcher-982-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1448654 INFO (jetty-launcher-982-thread-2) [n:127.0.0.1:46559_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1448687 INFO (jetty-launcher-982-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45875_solr
[junit4] 2> 1448773 INFO (jetty-launcher-982-thread-2) [n:127.0.0.1:46559_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46559_solr
[junit4] 2> 1448773 INFO (jetty-launcher-982-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer Overseer (id=72107581740744712-127.0.0.1:45875_solr-n_0000000000) starting
[junit4] 2> 1448842 INFO (zkCallback-1006-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1448931 INFO (zkCallback-1008-thread-1-processing-n:127.0.0.1:46559_solr) [n:127.0.0.1:46559_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1448964 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 1448964 DEBUG (OverseerAutoScalingTriggerThread-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1449053 DEBUG (OverseerAutoScalingTriggerThread-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1449106 DEBUG (OverseerAutoScalingTriggerThread-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:46559_solr]
[junit4] 2> 1449106 DEBUG (OverseerAutoScalingTriggerThread-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1449106 DEBUG (OverseerAutoScalingTriggerThread-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1449143 INFO (jetty-launcher-982-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45875_solr
[junit4] 2> 1449168 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:45875_solr"} current state version: 0
[junit4] 2> 1449168 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:45875_solr
[junit4] 2> 1449168 INFO (zkCallback-1008-thread-1-processing-n:127.0.0.1:46559_solr) [n:127.0.0.1:46559_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1449168 INFO (zkCallback-1006-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1449204 DEBUG (OverseerAutoScalingTriggerThread-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1449224 DEBUG (OverseerAutoScalingTriggerThread-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1449253 DEBUG (ScheduledTrigger-2136-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1449436 INFO (jetty-launcher-982-thread-2) [n:127.0.0.1:46559_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46559.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bcfe0f8
[junit4] 2> 1449489 INFO (jetty-launcher-982-thread-2) [n:127.0.0.1:46559_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46559.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bcfe0f8
[junit4] 2> 1449489 INFO (jetty-launcher-982-thread-2) [n:127.0.0.1:46559_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46559.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bcfe0f8
[junit4] 2> 1449490 INFO (jetty-launcher-982-thread-2) [n:127.0.0.1:46559_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/.
[junit4] 2> 1449601 INFO (jetty-launcher-982-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45875.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bcfe0f8
[junit4] 2> 1449710 INFO (jetty-launcher-982-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45875.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bcfe0f8
[junit4] 2> 1449710 INFO (jetty-launcher-982-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45875.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bcfe0f8
[junit4] 2> 1449711 INFO (jetty-launcher-982-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/.
[junit4] 2> 1449819 INFO (jetty-launcher-982-thread-3) [n:127.0.0.1:42339_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1449821 INFO (jetty-launcher-982-thread-3) [n:127.0.0.1:42339_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1449839 INFO (jetty-launcher-982-thread-3) [n:127.0.0.1:42339_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42339_solr
[junit4] 2> 1449875 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:42339_solr"} current state version: 0
[junit4] 2> 1449875 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:42339_solr
[junit4] 2> 1449875 INFO (zkCallback-1008-thread-1-processing-n:127.0.0.1:46559_solr) [n:127.0.0.1:46559_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1449876 INFO (zkCallback-1006-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1449980 INFO (zkCallback-1004-thread-1-processing-n:127.0.0.1:42339_solr) [n:127.0.0.1:42339_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1450273 INFO (jetty-launcher-982-thread-3) [n:127.0.0.1:42339_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42339.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bcfe0f8
[junit4] 2> 1450310 DEBUG (ScheduledTrigger-2136-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1450327 INFO (jetty-launcher-982-thread-3) [n:127.0.0.1:42339_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42339.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bcfe0f8
[junit4] 2> 1450327 INFO (jetty-launcher-982-thread-3) [n:127.0.0.1:42339_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42339.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bcfe0f8
[junit4] 2> 1450328 INFO (jetty-launcher-982-thread-3) [n:127.0.0.1:42339_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/.
[junit4] 2> 1450792 INFO (zkConnectionManagerCallback-1017-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1451023 INFO (zkConnectionManagerCallback-1021-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1451047 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[F0965BF548E1B463]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1451047 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[F0965BF548E1B463]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46513/solr ready
[junit4] 2> 1451137 INFO (qtp1670315217-5506) [n:127.0.0.1:46559_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1451214 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
[junit4] 2> 1451217 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/protwords.txt
[junit4] 2> 1451281 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
[junit4] 2> 1451311 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/solrconfig.xml
[junit4] 2> 1451313 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/synonyms.txt
[junit4] 2> 1451314 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/stopwords.txt
[junit4] 2> 1451329 DEBUG (ScheduledTrigger-2136-thread-2) [n:127.0.0.1:45875_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1451475 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/params.json
[junit4] 2> 1451476 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_it.txt
[junit4] 2> 1451494 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_no.txt
[junit4] 2> 1451496 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hy.txt
[junit4] 2> 1451497 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stemdict_nl.txt
[junit4] 2> 1451511 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ca.txt
[junit4] 2> 1451513 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_id.txt
[junit4] 2> 1451531 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ga.txt
[junit4] 2> 1451533 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_da.txt
[junit4] 2> 1451534 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hi.txt
[junit4] 2> 1451568 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ja.txt
[junit4] 2> 1451569 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_pt.txt
[junit4] 2> 1451570 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hu.txt
[junit4] 2> 1451614 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_el.txt
[junit4] 2> 1451714 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ru.txt
[junit4] 2> 1451721 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_tr.txt
[junit4] 2> 1451723 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ar.txt
[junit4] 2> 1451724 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/userdict_ja.txt
[junit4] 2> 1451725 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_it.txt
[junit4] 2> 1451742 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_eu.txt
[junit4] 2> 1451743 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_cz.txt
[junit4] 2> 1451744 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stoptags_ja.txt
[junit4] 2> 1451758 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_fr.txt
[junit4] 2> 1451759 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fa.txt
[junit4] 2> 1451760 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_de.txt
[junit4] 2> 1451761 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ga.txt
[junit4] 2> 1451876 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ca.txt
[junit4] 2> 1451891 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_nl.txt
[junit4] 2> 1451907 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_bg.txt
[junit4] 2> 1451923 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_es.txt
[junit4] 2> 1451924 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_en.txt
[junit4] 2> 1451940 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_lv.txt
[junit4] 2> 1452067 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_th.txt
[junit4] 2> 1452185 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ro.txt
[junit4] 2> 1452233 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fr.txt
[junit4] 2> 1452287 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_sv.txt
[junit4] 2> 1452289 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fi.txt
[junit4] 2> 1452302 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/hyphenations_ga.txt
[junit4] 2> 1452320 INFO (OverseerThreadFactory-2139-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_gl.txt
[junit4] 2> 1452329 DEBUG (ScheduledTrigger-2136-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1452433 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "numShards":"1",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 1452433 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
[junit4] 2> 1452450 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
[junit4] 2> 1452551 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 1452551 INFO (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1452666 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 1452666 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 1452666 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:45875/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 1452666 INFO (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:45875/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1452713 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 1452713 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:45875/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 1452747 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42339/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 1452747 INFO (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42339/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1452748 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:45875/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 1452749 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:45875/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "base_url":"http://127.0.0.1:42339/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 1452905 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
[junit4] 2> 1453090 INFO (qtp1670315217-5510) [n:127.0.0.1:46559_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1453091 INFO (qtp1670315217-5510) [n:127.0.0.1:46559_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1453109 INFO (qtp1968489003-5498) [n:127.0.0.1:42339_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n3&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1453109 INFO (qtp1968489003-5498) [n:127.0.0.1:42339_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1453126 INFO (qtp718397030-5487) [n:127.0.0.1:45875_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1453126 INFO (qtp718397030-5487) [n:127.0.0.1:45875_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1453208 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:45875/solr",
[junit4] 2> "node_name":"127.0.0.1:45875_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1453208 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:45875/solr",
[junit4] 2> "node_name":"127.0.0.1:45875_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 1453208 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"http://127.0.0.1:45875/solr","node_name":"127.0.0.1:45875_solr","state":"down","type":"NRT"}
[junit4] 2> 1453208 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:45875/solr",
[junit4] 2> "node_name":"127.0.0.1:45875_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "base_url":"http://127.0.0.1:42339/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 1453260 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "node_name":"127.0.0.1:46559_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1453260 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "node_name":"127.0.0.1:46559_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 1453260 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node4:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"http://127.0.0.1:46559/solr","node_name":"127.0.0.1:46559_solr","state":"down","type":"NRT"}
[junit4] 2> 1453261 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "node_name":"127.0.0.1:46559_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:45875/solr",
[junit4] 2> "node_name":"127.0.0.1:45875_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "base_url":"http://127.0.0.1:42339/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 1453272 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42339/solr",
[junit4] 2> "node_name":"127.0.0.1:42339_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1453272 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42339/solr",
[junit4] 2> "node_name":"127.0.0.1:42339_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 1453272 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3","base_url":"http://127.0.0.1:42339/solr","node_name":"127.0.0.1:42339_solr","state":"down","type":"NRT"}
[junit4] 2> 1453272 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node4":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:46559/solr",
[junit4] 2> "node_name":"127.0.0.1:46559_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:45875/solr",
[junit4] 2> "node_name":"127.0.0.1:45875_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n3",
[junit4] 2> "base_url":"http://127.0.0.1:42339/solr",
[junit4] 2> "node_name":"127.0.0.1:42339_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 1453344 DEBUG (ScheduledTrigger-2136-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1453381 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
[junit4] 2> 1453381 INFO (zkCallback-1004-thread-1-processing-n:127.0.0.1:42339_solr) [n:127.0.0.1:42339_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1453381 INFO (zkCallback-1008-thread-1-processing-n:127.0.0.1:46559_solr) [n:127.0.0.1:46559_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1453381 INFO (zkCallback-1006-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1453382 INFO (zkCallback-1006-thread-2-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1453383 INFO (zkCallback-1008-thread-2-processing-n:127.0.0.1:46559_solr) [n:127.0.0.1:46559_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1453383 INFO (zkCallback-1004-thread-2-processing-n:127.0.0.1:42339_solr) [n:127.0.0.1:42339_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [3])
[junit4] 2> 1454395 DEBUG (ScheduledTrigger-2136-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1454397 INFO (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 1454447 INFO (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 1454447 INFO (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 1454448 WARN (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/extraction/lib
[junit4] 2> 1454486 WARN (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist
[junit4] 2> 1454486 INFO (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 1454519 INFO (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 1454520 WARN (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
[junit4] 2> 1454520 WARN (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 1454520 WARN (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/clustering/lib
[junit4] 2> 1454521 WARN (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 1454521 WARN (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/langid/lib
[junit4] 2> 1454521 WARN (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 1454521 WARN (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/velocity/lib
[junit4] 2> 1454521 WARN (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 1454484 INFO (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 1454559 WARN (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/clustering/lib
[junit4] 2> 1454559 WARN (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist
[junit4] 2> 1454559 WARN (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/langid/lib
[junit4] 2> 1454559 WARN (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist
[junit4] 2> 1454560 WARN (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../contrib/velocity/lib
[junit4] 2> 1454578 INFO (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1454577 WARN (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
[junit4] 2> 1454613 WARN (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 1454613 WARN (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
[junit4] 2> 1454613 WARN (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 1454614 WARN (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/langid/lib
[junit4] 2> 1454614 WARN (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-langid-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 1454614 WARN (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/velocity/lib
[junit4] 2> 1454614 WARN (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node2/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 1454629 WARN (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist filtered by solr-velocity-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001/tempDir-001/node3/AutoscalingHistoryHandlerTest_collection_shard1_replica_n3/../../../../dist
[junit4] 2> 1454653 INFO (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1454741 INFO (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1454798 INFO (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.s.IndexSchema [AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] Schema name=default-config
[junit4] 2> 1454830 INFO (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.s.IndexSchema [AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] Schema name=default-config
[junit4] 2> 1454835 INFO (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.s.IndexSchema [AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] Schema name=default-config
[junit4] 2> 1455557 DEBUG (ScheduledTrigger-2136-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1456251 INFO (qtp718397030-5487) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
[junit4] 2> 1456581 DEBUG (ScheduledTrigger-2136-thread-1) [n:127.0.0.1:45875_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1456703 INFO (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
[junit4] 2> 1457088 INFO (qtp1670315217-5510) [n:127.0.0.1:46559_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node4 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema default-config/1.6 with uniqueid field id
[junit4] 2> 1457223 INFO (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'AutoscalingHistoryHandlerTest_collection_shard1_replica_n3' using configuration from collection AutoscalingHistoryHandlerTest_collection, trusted=true
[junit4] 2> 1457224 INFO (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42339.solr.core.AutoscalingHistoryHandlerTest_collection.shard1.replica_n3' (registry 'solr.core.AutoscalingHistoryHandlerTest_collection.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3bcfe0f8
[junit4] 2> 1457224 INFO (qtp1968489003-5498) [n:127.0.0.1:42339_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node6 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n3] o.a.s.c.
[...truncated too long message...]
ollection.shard1.replica_n7, service url = null, agent id = null] for registry solr.core.AutoscalingHistoryHandlerTest_collection.shard1.replica_n7 / com.codahale.metrics.MetricRegistry@4d77366d
[junit4] 2> 1506256 INFO (coreCloseExecutor-2201-thread-2) [n:127.0.0.1:42339_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=930477246
[junit4] 2> 1506277 INFO (coreCloseExecutor-2200-thread-1) [n:127.0.0.1:45875_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.AutoscalingHistoryHandlerTest_collection.shard1.leader, tag=1208797840
[junit4] 2> 1506303 INFO (jetty-closer-983-thread-1) [ ] o.a.s.c.Overseer Overseer (id=72107581740744712-127.0.0.1:45875_solr-n_0000000000) closing
[junit4] 2> 1506305 DEBUG (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/.system/state.json version: 20
[junit4] 2> 1506305 INFO (jetty-closer-983-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1225e503{/solr,null,UNAVAILABLE}
[junit4] 2> 1506306 INFO (jetty-closer-983-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1506384 INFO (coreCloseExecutor-2202-thread-2) [n:127.0.0.1:43997_solr c:.system s:shard1 r:core_node8 x:.system_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=1952929017
[junit4] 2> 1506410 INFO (zkCallback-1006-thread-1-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1506410 INFO (zkCallback-1006-thread-2-processing-n:127.0.0.1:45875_solr) [n:127.0.0.1:45875_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1506425 INFO (OverseerStateUpdate-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:45875_solr
[junit4] 2> 1506425 INFO (zkCallback-1028-thread-4-processing-n:127.0.0.1:43997_solr) [n:127.0.0.1:43997_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1506425 INFO (zkCallback-1028-thread-2-processing-n:127.0.0.1:43997_solr) [n:127.0.0.1:43997_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1506426 INFO (coreCloseExecutor-2202-thread-1) [n:127.0.0.1:43997_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node8 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.AutoscalingHistoryHandlerTest_collection.shard1.leader, tag=311621235
[junit4] 2> 1506427 DEBUG (jetty-closer-983-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1506428 WARN (OverseerAutoScalingTriggerThread-72107581740744712-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1506491 INFO (zkCallback-1028-thread-4-processing-n:127.0.0.1:43997_solr) [n:127.0.0.1:43997_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43997_solr
[junit4] 2> 1506507 INFO (jetty-closer-983-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2c3d9faf{/solr,null,UNAVAILABLE}
[junit4] 2> 1506518 INFO (jetty-closer-983-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2ab19747{/solr,null,UNAVAILABLE}
[junit4] 2> 1506534 INFO (jetty-closer-983-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1506567 INFO (jetty-closer-983-thread-3) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1506569 ERROR (SUITE-AutoscalingHistoryHandlerTest-seed#[F0965BF548E1B463]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1506569 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[F0965BF548E1B463]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46513 46513
[junit4] 2> 1506636 INFO (Thread-1564) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46513 46513
[junit4] 2> 1506637 WARN (Thread-1564) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/collections/.system/terms/shard1
[junit4] 2> 17 /solr/collections/AutoscalingHistoryHandlerTest_collection/terms/shard1
[junit4] 2> 13 /solr/aliases.json
[junit4] 2> 13 /solr/clusterprops.json
[junit4] 2> 5 /solr/configs/.system
[junit4] 2> 5 /solr/configs/.system/managed-schema
[junit4] 2> 4 /solr/security.json
[junit4] 2> 4 /solr/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
[junit4] 2> 4 /solr/configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 68 /solr/collections/.system/state.json
[junit4] 2> 40 /solr/collections/AutoscalingHistoryHandlerTest_collection/state.json
[junit4] 2> 13 /solr/clusterstate.json
[junit4] 2> 3 /solr/overseer_elect/election/72107581740744712-127.0.0.1:45875_solr-n_0000000000
[junit4] 2> 2 /solr/collections/AutoscalingHistoryHandlerTest_collection/leader_elect/shard1/election/72107581740744712-core_node5-n_0000000001
[junit4] 2> 2 /solr/collections/.system/leader_elect/shard1/election/72107581740744713-core_node6-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 13 /solr/live_nodes
[junit4] 2> 13 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_F0965BF548E1B463-001
[junit4] 2> Feb 12, 2018 8:50:10 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {action_s=FST50, event.source_s=PostingsFormat(name=MockRandom), type=FST50, before.actions_ss=PostingsFormat(name=MockRandom), event.property._enqueue_time__ss=FST50, after.actions_ss=PostingsFormat(name=LuceneFixedGap), stage_s=FST50, event.id_s=BlockTreeOrds(blocksize=128), event.property.eventTimes_ss=FST50, source_s=BlockTreeOrds(blocksize=128), event.type_s=FST50, responses_ts=PostingsFormat(name=LuceneFixedGap), id=BlockTreeOrds(blocksize=128), collections_ss=FST50, operations.params_ts=FST50, event.property.nodeNames_ss=FST50}, docValues:{_version_=DocValuesFormat(name=Asserting), event.time_l=DocValuesFormat(name=Asserting), timestamp=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=158, maxMBSortInHeap=5.4182082603962085, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@38f9bb57), locale=ar-TN, timezone=Europe/Paris
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=142622624,total=523239424
[junit4] 2> NOTE: All tests run in this JVM: [TestTrieFacet, TestSystemCollAutoCreate, TestComplexPhraseLeadingWildcard, TestDocumentBuilder, TestRandomFlRTGCloud, TestSchemaVersionResource, TestRandomFaceting, UUIDUpdateProcessorFallbackTest, TestQuerySenderNoQuery, TestPhraseSuggestions, ExternalFileFieldSortTest, TestComponentsName, TestLegacyField, DeleteLastCustomShardedReplicaTest, SolrIndexConfigTest, TestMaxScoreQueryParser, TestHdfsUpdateLog, TestRemoteStreaming, TestSmileRequest, TestFieldCacheSortRandom, TestDocSet, BasicDistributedZkTest, SolrShardReporterTest, TestLockTree, JvmMetricsTest, TestLeaderElectionWithEmptyReplica, CurrencyFieldTypeTest, TestPrepRecovery, EnumFieldTest, TestNamedUpdateProcessors, MetricUtilsTest, ChaosMonkeySafeLeaderWithPullReplicasTest, PluginInfoTest, SuggesterWFSTTest, TestPayloadScoreQParserPlugin, TestCSVLoader, TestFieldSortValues, AssignBackwardCompatibilityTest, DirectUpdateHandlerTest, TestShortCircuitedRequests, SpellCheckCollatorWithCollapseTest, TestGroupingSearch, TestLuceneIndexBackCompat, TestFileDictionaryLookup, StatsReloadRaceTest, OpenCloseCoreStressTest, DistributedDebugComponentTest, HDFSCollectionsAPITest, TestBackupRepositoryFactory, TestMaxTokenLenTokenizer, TestCollationField, FieldAnalysisRequestHandlerTest, ActionThrottleTest, CustomCollectionTest, ConfigureRecoveryStrategyTest, BadCopyFieldTest, SmileWriterTest, TestExtendedDismaxParser, TestMergePolicyConfig, AutoscalingHistoryHandlerTest]
[junit4] Completed [158/776 (1!)] on J0 in 60.65s, 1 test, 1 failure <<< FAILURES!
[...truncated 50439 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2314 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2314/
1 tests failed.
FAILED: org.apache.solr.cloud.TestUtilizeNode.test
Error Message:
no replica should be present in 127.0.0.1:37541_solr
Stack Trace:
java.lang.AssertionError: no replica should be present in 127.0.0.1:37541_solr
at __randomizedtesting.SeedInfo.seed([4BDCD6082D33F991:C388E9D283CF9469]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.TestUtilizeNode.test(TestUtilizeNode.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12254 lines...]
[junit4] Suite: org.apache.solr.cloud.TestUtilizeNode
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/init-core-data-001
[junit4] 2> 177499 INFO (SUITE-TestUtilizeNode-seed#[4BDCD6082D33F991]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 177500 INFO (SUITE-TestUtilizeNode-seed#[4BDCD6082D33F991]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 177500 INFO (SUITE-TestUtilizeNode-seed#[4BDCD6082D33F991]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 177501 INFO (SUITE-TestUtilizeNode-seed#[4BDCD6082D33F991]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001
[junit4] 2> 177501 INFO (SUITE-TestUtilizeNode-seed#[4BDCD6082D33F991]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 177501 INFO (Thread-1786) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 177501 INFO (Thread-1786) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 177504 ERROR (Thread-1786) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 177601 INFO (SUITE-TestUtilizeNode-seed#[4BDCD6082D33F991]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:54329
[junit4] 2> 177604 INFO (zkConnectionManagerCallback-416-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177619 INFO (jetty-launcher-413-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 177620 INFO (jetty-launcher-413-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 177619 INFO (jetty-launcher-413-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 177623 INFO (jetty-launcher-413-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 177623 INFO (jetty-launcher-413-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 177623 INFO (jetty-launcher-413-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 177624 INFO (jetty-launcher-413-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 177624 INFO (jetty-launcher-413-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 177624 INFO (jetty-launcher-413-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 177624 INFO (jetty-launcher-413-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 177625 INFO (jetty-launcher-413-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5ac581c4{/solr,null,AVAILABLE}
[junit4] 2> 177627 INFO (jetty-launcher-413-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 177627 INFO (jetty-launcher-413-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 177627 INFO (jetty-launcher-413-thread-4) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 177628 INFO (jetty-launcher-413-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@678af4de{/solr,null,AVAILABLE}
[junit4] 2> 177628 INFO (jetty-launcher-413-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 177628 INFO (jetty-launcher-413-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 177628 INFO (jetty-launcher-413-thread-3) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 177629 INFO (jetty-launcher-413-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@ffd19b1{SSL,[ssl, http/1.1]}{127.0.0.1:48720}
[junit4] 2> 177629 INFO (jetty-launcher-413-thread-2) [ ] o.e.j.s.Server Started @179871ms
[junit4] 2> 177629 INFO (jetty-launcher-413-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48720}
[junit4] 2> 177629 INFO (jetty-launcher-413-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7e91b4ae{/solr,null,AVAILABLE}
[junit4] 2> 177629 ERROR (jetty-launcher-413-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 177629 INFO (jetty-launcher-413-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 177629 INFO (jetty-launcher-413-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 177629 INFO (jetty-launcher-413-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 177629 INFO (jetty-launcher-413-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T23:13:43.364Z
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@738fdbe0{SSL,[ssl, http/1.1]}{127.0.0.1:39636}
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-1) [ ] o.e.j.s.Server Started @179872ms
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39636}
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@49e34cd4{/solr,null,AVAILABLE}
[junit4] 2> 177630 ERROR (jetty-launcher-413-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1961d1e4{SSL,[ssl, http/1.1]}{127.0.0.1:33884}
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T23:13:43.365Z
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-3) [ ] o.e.j.s.Server Started @179872ms
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33884}
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@652f236d{SSL,[ssl, http/1.1]}{127.0.0.1:53641}
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-4) [ ] o.e.j.s.Server Started @179873ms
[junit4] 2> 177630 INFO (jetty-launcher-413-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53641}
[junit4] 2> 177631 ERROR (jetty-launcher-413-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 177631 INFO (jetty-launcher-413-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 177631 INFO (jetty-launcher-413-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 177631 INFO (jetty-launcher-413-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 177631 ERROR (jetty-launcher-413-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 177631 INFO (jetty-launcher-413-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T23:13:43.366Z
[junit4] 2> 177631 INFO (jetty-launcher-413-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 177631 INFO (jetty-launcher-413-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 177631 INFO (jetty-launcher-413-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 177631 INFO (jetty-launcher-413-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T23:13:43.366Z
[junit4] 2> 177634 INFO (zkConnectionManagerCallback-420-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177634 INFO (zkConnectionManagerCallback-418-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177634 INFO (jetty-launcher-413-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 177634 INFO (jetty-launcher-413-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 177635 INFO (zkConnectionManagerCallback-424-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177635 INFO (jetty-launcher-413-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 177639 INFO (zkConnectionManagerCallback-422-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177640 INFO (jetty-launcher-413-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 177645 INFO (jetty-launcher-413-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 177646 INFO (jetty-launcher-413-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 177650 INFO (jetty-launcher-413-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 177659 INFO (jetty-launcher-413-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 177666 INFO (zkConnectionManagerCallback-429-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177666 INFO (zkConnectionManagerCallback-432-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177672 INFO (zkConnectionManagerCallback-436-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177674 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10457318b4c0006, likely client has closed socket
[junit4] 2> 177674 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10457318b4c0005, likely client has closed socket
[junit4] 2> 177676 INFO (zkConnectionManagerCallback-442-thread-1-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177677 INFO (zkConnectionManagerCallback-440-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177678 INFO (zkConnectionManagerCallback-444-thread-1-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177697 INFO (zkConnectionManagerCallback-446-thread-1-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177698 INFO (zkConnectionManagerCallback-450-thread-1-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 177812 INFO (jetty-launcher-413-thread-1) [n:127.0.0.1:39636_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 177815 INFO (jetty-launcher-413-thread-2) [n:127.0.0.1:48720_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 177815 INFO (jetty-launcher-413-thread-1) [n:127.0.0.1:39636_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39636_solr
[junit4] 2> 177816 INFO (jetty-launcher-413-thread-4) [n:127.0.0.1:53641_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 177816 INFO (jetty-launcher-413-thread-2) [n:127.0.0.1:48720_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48720_solr
[junit4] 2> 177822 INFO (jetty-launcher-413-thread-1) [n:127.0.0.1:39636_solr ] o.a.s.c.Overseer Overseer (id=73279364246798347-127.0.0.1:39636_solr-n_0000000000) starting
[junit4] 2> 177824 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 177825 INFO (zkCallback-441-thread-1-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 177829 INFO (zkCallback-449-thread-1-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 177830 INFO (jetty-launcher-413-thread-3) [n:127.0.0.1:33884_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 177831 INFO (jetty-launcher-413-thread-4) [n:127.0.0.1:53641_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53641_solr
[junit4] 2> 177832 INFO (jetty-launcher-413-thread-3) [n:127.0.0.1:33884_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33884_solr
[junit4] 2> 177833 INFO (zkCallback-449-thread-1-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 177833 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 177843 INFO (zkCallback-445-thread-1-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 177843 INFO (zkCallback-445-thread-2-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 177844 INFO (zkCallback-441-thread-1-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 177844 INFO (zkCallback-449-thread-2-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 177844 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 177851 INFO (jetty-launcher-413-thread-1) [n:127.0.0.1:39636_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39636_solr
[junit4] 2> 177858 INFO (zkCallback-449-thread-2-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 177858 INFO (zkCallback-441-thread-1-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 177858 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 177858 INFO (zkCallback-445-thread-2-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 177949 INFO (jetty-launcher-413-thread-3) [n:127.0.0.1:33884_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33884.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 177964 INFO (jetty-launcher-413-thread-1) [n:127.0.0.1:39636_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39636.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 177979 INFO (jetty-launcher-413-thread-3) [n:127.0.0.1:33884_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33884.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 177979 INFO (jetty-launcher-413-thread-3) [n:127.0.0.1:33884_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33884.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 177982 INFO (jetty-launcher-413-thread-3) [n:127.0.0.1:33884_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node3/.
[junit4] 2> 177985 INFO (jetty-launcher-413-thread-4) [n:127.0.0.1:53641_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53641.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 177985 INFO (jetty-launcher-413-thread-2) [n:127.0.0.1:48720_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48720.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 177991 INFO (jetty-launcher-413-thread-1) [n:127.0.0.1:39636_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39636.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 177991 INFO (jetty-launcher-413-thread-1) [n:127.0.0.1:39636_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39636.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 177992 INFO (jetty-launcher-413-thread-1) [n:127.0.0.1:39636_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node1/.
[junit4] 2> 177996 INFO (jetty-launcher-413-thread-4) [n:127.0.0.1:53641_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53641.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 177997 INFO (jetty-launcher-413-thread-4) [n:127.0.0.1:53641_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53641.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 177998 INFO (jetty-launcher-413-thread-4) [n:127.0.0.1:53641_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node4/.
[junit4] 2> 177999 INFO (jetty-launcher-413-thread-2) [n:127.0.0.1:48720_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48720.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 177999 INFO (jetty-launcher-413-thread-2) [n:127.0.0.1:48720_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48720.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 178000 INFO (jetty-launcher-413-thread-2) [n:127.0.0.1:48720_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node2/.
[junit4] 2> 178126 INFO (zkConnectionManagerCallback-458-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 178129 INFO (zkConnectionManagerCallback-462-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 178129 INFO (SUITE-TestUtilizeNode-seed#[4BDCD6082D33F991]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 178130 INFO (SUITE-TestUtilizeNode-seed#[4BDCD6082D33F991]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:54329/solr ready
[junit4] 2> 178163 INFO (qtp1721654900-3195) [n:127.0.0.1:48720_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 178173 INFO (qtp1721654900-3195) [n:127.0.0.1:48720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=9
[junit4] 2> 178199 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 178200 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 178202 INFO (zkConnectionManagerCallback-465-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 178204 INFO (qtp1721654900-3197) [n:127.0.0.1:48720_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&name=utilizenodecoll&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 178206 INFO (OverseerThreadFactory-801-thread-2-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection utilizenodecoll
[junit4] 2> 178206 INFO (OverseerCollectionConfigSetProcessor-73279364246798347-127.0.0.1:39636_solr-n_0000000000) [n:127.0.0.1:39636_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 178315 INFO (OverseerStateUpdate-73279364246798347-127.0.0.1:39636_solr-n_0000000000) [n:127.0.0.1:39636_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"utilizenodecoll_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:33884/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 178322 INFO (OverseerStateUpdate-73279364246798347-127.0.0.1:39636_solr-n_0000000000) [n:127.0.0.1:39636_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"utilizenodecoll_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:53641/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 178323 INFO (OverseerStateUpdate-73279364246798347-127.0.0.1:39636_solr-n_0000000000) [n:127.0.0.1:39636_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"utilizenodecoll_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:48720/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 178325 INFO (OverseerStateUpdate-73279364246798347-127.0.0.1:39636_solr-n_0000000000) [n:127.0.0.1:39636_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"utilizenodecoll_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:39636/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 178574 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard2_replica_n4&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 178579 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 178591 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard1_replica_n1&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 178592 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 178608 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard2_replica_n6&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 178608 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 178610 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard1_replica_n3&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 178610 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 178715 INFO (zkCallback-449-thread-1-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 178715 INFO (zkCallback-449-thread-2-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 178715 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 178715 INFO (zkCallback-445-thread-2-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 178715 INFO (zkCallback-441-thread-1-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 178715 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 178715 INFO (zkCallback-445-thread-1-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 178715 INFO (zkCallback-441-thread-2-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 179612 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 179614 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 179623 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.s.IndexSchema [utilizenodecoll_shard2_replica_n4] Schema name=cloud-dynamic
[junit4] 2> 179624 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 179625 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.s.IndexSchema [utilizenodecoll_shard1_replica_n1] Schema name=cloud-dynamic
[junit4] 2> 179637 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 179642 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.s.IndexSchema [utilizenodecoll_shard2_replica_n6] Schema name=cloud-dynamic
[junit4] 2> 179645 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.s.IndexSchema [utilizenodecoll_shard1_replica_n3] Schema name=cloud-dynamic
[junit4] 2> 179665 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 179665 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'utilizenodecoll_shard2_replica_n4' using configuration from collection utilizenodecoll, trusted=true
[junit4] 2> 179666 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48720.solr.core.utilizenodecoll.shard2.replica_n4' (registry 'solr.core.utilizenodecoll.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 179666 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 179666 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SolrCore [[utilizenodecoll_shard2_replica_n4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node2/utilizenodecoll_shard2_replica_n4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node2/./utilizenodecoll_shard2_replica_n4/data/]
[junit4] 2> 179666 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 179666 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'utilizenodecoll_shard1_replica_n1' using configuration from collection utilizenodecoll, trusted=true
[junit4] 2> 179667 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33884.solr.core.utilizenodecoll.shard1.replica_n1' (registry 'solr.core.utilizenodecoll.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 179667 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 179667 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SolrCore [[utilizenodecoll_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node3/utilizenodecoll_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node3/./utilizenodecoll_shard1_replica_n1/data/]
[junit4] 2> 179669 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 179669 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'utilizenodecoll_shard2_replica_n6' using configuration from collection utilizenodecoll, trusted=true
[junit4] 2> 179673 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 179673 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'utilizenodecoll_shard1_replica_n3' using configuration from collection utilizenodecoll, trusted=true
[junit4] 2> 179674 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53641.solr.core.utilizenodecoll.shard1.replica_n3' (registry 'solr.core.utilizenodecoll.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 179675 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 179675 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.c.SolrCore [[utilizenodecoll_shard1_replica_n3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node4/utilizenodecoll_shard1_replica_n3], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node4/./utilizenodecoll_shard1_replica_n3/data/]
[junit4] 2> 179690 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39636.solr.core.utilizenodecoll.shard2.replica_n6' (registry 'solr.core.utilizenodecoll.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 179690 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 179690 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.c.SolrCore [[utilizenodecoll_shard2_replica_n6] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node1/utilizenodecoll_shard2_replica_n6], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node1/./utilizenodecoll_shard2_replica_n6/data/]
[junit4] 2> 179733 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 179733 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 179734 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 179734 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 179736 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2dc3521a[utilizenodecoll_shard1_replica_n1] main]
[junit4] 2> 179737 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 179737 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 179737 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 179737 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 179738 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 179738 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 179738 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592148178211176448
[junit4] 2> 179738 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 179738 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 179738 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 179753 INFO (searcherExecutor-811-thread-1-processing-n:127.0.0.1:33884_solr x:utilizenodecoll_shard1_replica_n1 s:shard1 c:utilizenodecoll r:core_node2) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SolrCore [utilizenodecoll_shard1_replica_n1] Registered new searcher Searcher@2dc3521a[utilizenodecoll_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 179753 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 179753 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 179753 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@3822d224[utilizenodecoll_shard2_replica_n4] main]
[junit4] 2> 179754 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@15621a84[utilizenodecoll_shard2_replica_n6] main]
[junit4] 2> 179755 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 179755 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 179756 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 179756 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 179756 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 179756 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 179757 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592148178231099392
[junit4] 2> 179757 INFO (searcherExecutor-810-thread-1-processing-n:127.0.0.1:48720_solr x:utilizenodecoll_shard2_replica_n4 s:shard2 c:utilizenodecoll r:core_node7) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SolrCore [utilizenodecoll_shard2_replica_n4] Registered new searcher Searcher@3822d224[utilizenodecoll_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 179757 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592148178231099392
[junit4] 2> 179762 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 179764 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 179766 INFO (searcherExecutor-812-thread-1-processing-n:127.0.0.1:39636_solr x:utilizenodecoll_shard2_replica_n6 s:shard2 c:utilizenodecoll r:core_node8) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.c.SolrCore [utilizenodecoll_shard2_replica_n6] Registered new searcher Searcher@15621a84[utilizenodecoll_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 179766 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 179766 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 179767 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 179767 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 179769 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@5bcc7d01[utilizenodecoll_shard1_replica_n3] main]
[junit4] 2> 179769 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 179770 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 179770 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 179771 INFO (searcherExecutor-813-thread-1-processing-n:127.0.0.1:53641_solr x:utilizenodecoll_shard1_replica_n3 s:shard1 c:utilizenodecoll r:core_node5) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.c.SolrCore [utilizenodecoll_shard1_replica_n3] Registered new searcher Searcher@5bcc7d01[utilizenodecoll_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 179771 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592148178245779456
[junit4] 2> 179866 INFO (zkCallback-449-thread-1-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 179866 INFO (zkCallback-449-thread-2-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 179866 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 179866 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 179866 INFO (zkCallback-445-thread-1-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 179866 INFO (zkCallback-445-thread-2-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 179866 INFO (zkCallback-441-thread-1-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 179866 INFO (zkCallback-441-thread-2-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180262 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 180262 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 180262 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:33884/solr/utilizenodecoll_shard1_replica_n1/
[junit4] 2> 180263 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=utilizenodecoll_shard1_replica_n1 url=https://127.0.0.1:33884/solr START replicas=[https://127.0.0.1:53641/solr/utilizenodecoll_shard1_replica_n3/] nUpdates=100
[junit4] 2> 180265 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 180265 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 180265 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:48720/solr/utilizenodecoll_shard2_replica_n4/
[junit4] 2> 180266 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.PeerSync PeerSync: core=utilizenodecoll_shard2_replica_n4 url=https://127.0.0.1:48720/solr START replicas=[https://127.0.0.1:39636/solr/utilizenodecoll_shard2_replica_n6/] nUpdates=100
[junit4] 2> 180270 INFO (qtp1129421360-3210) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.c.S.Request [utilizenodecoll_shard1_replica_n3] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 180274 INFO (qtp451332405-3204) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.c.S.Request [utilizenodecoll_shard2_replica_n6] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 180563 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=utilizenodecoll_shard1_replica_n1 url=https://127.0.0.1:33884/solr DONE. We have no versions. sync failed.
[junit4] 2> 180563 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 180563 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 180563 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 180566 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.PeerSync PeerSync: core=utilizenodecoll_shard2_replica_n4 url=https://127.0.0.1:48720/solr DONE. We have no versions. sync failed.
[junit4] 2> 180566 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 180566 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 180566 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 180566 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:33884/solr/utilizenodecoll_shard1_replica_n1/ shard1
[junit4] 2> 180568 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:48720/solr/utilizenodecoll_shard2_replica_n4/ shard2
[junit4] 2> 180670 INFO (zkCallback-445-thread-1-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180670 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180670 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180670 INFO (zkCallback-441-thread-1-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180670 INFO (zkCallback-449-thread-2-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180670 INFO (zkCallback-449-thread-1-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180670 INFO (zkCallback-441-thread-2-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180670 INFO (zkCallback-445-thread-2-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180717 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 180719 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr c:utilizenodecoll s:shard1 r:core_node2 x:utilizenodecoll_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard1_replica_n1&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2127
[junit4] 2> 180719 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 180720 INFO (qtp1721654900-3198) [n:127.0.0.1:48720_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard2_replica_n4&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2146
[junit4] 2> 180767 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard2_replica_n6&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2159
[junit4] 2> 180778 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr c:utilizenodecoll s:shard1 r:core_node5 x:utilizenodecoll_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard1_replica_n3&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2167
[junit4] 2> 180784 INFO (qtp1721654900-3197) [n:127.0.0.1:48720_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 180879 INFO (zkCallback-441-thread-1-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180879 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180879 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180879 INFO (zkCallback-445-thread-3-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180879 INFO (zkCallback-449-thread-1-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180879 INFO (zkCallback-449-thread-2-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180879 INFO (zkCallback-445-thread-2-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 180879 INFO (zkCallback-441-thread-2-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 181784 INFO (qtp1721654900-3197) [n:127.0.0.1:48720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf1&name=utilizenodecoll&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3580
[junit4] 2> 181786 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T15:27:37-06:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 181788 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 181788 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 181788 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 181789 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4f60fb9f{/solr,null,AVAILABLE}
[junit4] 2> 181789 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@76e47726{SSL,[ssl, http/1.1]}{127.0.0.1:37541}
[junit4] 2> 181789 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.e.j.s.Server Started @184032ms
[junit4] 2> 181789 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37541}
[junit4] 2> 181790 ERROR (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 181791 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 181791 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 181791 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 181791 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T23:13:47.526Z
[junit4] 2> 181794 INFO (zkConnectionManagerCallback-467-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 181794 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 181802 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:54329/solr
[junit4] 2> 181803 INFO (zkConnectionManagerCallback-471-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 181805 INFO (zkConnectionManagerCallback-473-thread-1-processing-n:127.0.0.1:37541_solr) [n:127.0.0.1:37541_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 181811 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [n:127.0.0.1:37541_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 181813 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [n:127.0.0.1:37541_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 181814 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [n:127.0.0.1:37541_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37541_solr
[junit4] 2> 181815 INFO (zkCallback-461-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 181815 INFO (zkCallback-449-thread-2-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 181815 INFO (zkCallback-461-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 181815 INFO (zkCallback-441-thread-2-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 181815 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 181816 INFO (zkCallback-445-thread-2-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 181816 INFO (zkCallback-472-thread-1-processing-n:127.0.0.1:37541_solr) [n:127.0.0.1:37541_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 181876 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [n:127.0.0.1:37541_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37541.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 181885 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [n:127.0.0.1:37541_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37541.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 181885 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [n:127.0.0.1:37541_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37541.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 181886 INFO (TEST-TestUtilizeNode.test-seed#[4BDCD6082D33F991]) [n:127.0.0.1:37541_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node5/.
[junit4] 2> 181937 INFO (zkConnectionManagerCallback-477-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 181942 INFO (qtp1411202021-3224) [n:127.0.0.1:33884_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :utilizenode with params node=127.0.0.1:37541_solr&action=UTILIZENODE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 181944 INFO (OverseerCollectionConfigSetProcessor-73279364246798347-127.0.0.1:39636_solr-n_0000000000) [n:127.0.0.1:39636_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 181960 INFO (qtp1411202021-3223) [n:127.0.0.1:33884_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.utilizenodecoll.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=1
[junit4] 2> 181973 INFO (qtp1411202021-3225) [n:127.0.0.1:33884_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=9
[junit4] 2> 182002 INFO (qtp283626442-3342) [n:127.0.0.1:37541_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 182006 INFO (qtp451332405-3203) [n:127.0.0.1:39636_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.utilizenodecoll.shard2.replica_n6:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 182008 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 182012 INFO (qtp1721654900-3195) [n:127.0.0.1:48720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.utilizenodecoll.shard2.replica_n4:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 182013 INFO (qtp1721654900-3197) [n:127.0.0.1:48720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 182016 INFO (qtp1129421360-3214) [n:127.0.0.1:53641_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.utilizenodecoll.shard1.replica_n3:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 182017 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 182022 INFO (qtp1411202021-3223) [n:127.0.0.1:33884_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.utilizenodecoll.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 182024 INFO (qtp1411202021-3225) [n:127.0.0.1:33884_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 182025 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 182026 INFO (qtp451332405-3203) [n:127.0.0.1:39636_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.utilizenodecoll.shard2.replica_n6:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 182027 INFO (qtp451332405-3206) [n:127.0.0.1:39636_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 182028 INFO (qtp1721654900-3195) [n:127.0.0.1:48720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.utilizenodecoll.shard2.replica_n4:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 182029 INFO (qtp1721654900-3197) [n:127.0.0.1:48720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 182030 INFO (qtp1129421360-3214) [n:127.0.0.1:53641_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.utilizenodecoll.shard1.replica_n3:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 182031 INFO (qtp1129421360-3216) [n:127.0.0.1:53641_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 182038 INFO (OverseerThreadFactory-801-thread-3-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18642738436 sessionWrapper.createTime 18642738429671999, this.sessionWrapper.createTime 18642738429671999
[junit4] 2> 182038 INFO (OverseerThreadFactory-801-thread-3-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.a.c.MoveReplicaCmd Replica will be moved to node 127.0.0.1:37541_solr: core_node8:{"core":"utilizenodecoll_shard2_replica_n6","base_url":"https://127.0.0.1:39636/solr","node_name":"127.0.0.1:39636_solr","state":"active","type":"NRT"}
[junit4] 2> 182040 INFO (OverseerThreadFactory-801-thread-3-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:37541_solr for creating new replica
[junit4] 2> 182041 INFO (OverseerStateUpdate-73279364246798347-127.0.0.1:39636_solr-n_0000000000) [n:127.0.0.1:39636_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"addreplica",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"utilizenodecoll_shard2_replica_n9",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37541/solr",
[junit4] 2> "node_name":"127.0.0.1:37541_solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 182143 INFO (zkCallback-449-thread-1-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [5])
[junit4] 2> 182143 INFO (zkCallback-449-thread-2-processing-n:127.0.0.1:33884_solr) [n:127.0.0.1:33884_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [5])
[junit4] 2> 182143 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [5])
[junit4] 2> 182143 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:48720_solr) [n:127.0.0.1:48720_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [5])
[junit4] 2> 182143 INFO (zkCallback-441-thread-2-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [5])
[junit4] 2> 182143 INFO (zkCallback-441-thread-1-processing-n:127.0.0.1:53641_solr) [n:127.0.0.1:53641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [5])
[junit4] 2> 182144 INFO (zkCallback-445-thread-2-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [5])
[junit4] 2> 182144 INFO (zkCallback-445-thread-3-processing-n:127.0.0.1:39636_solr) [n:127.0.0.1:39636_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [5])
[junit4] 2> 182245 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node10&collection.configName=conf1&name=utilizenodecoll_shard2_replica_n9&action=CREATE&collection=utilizenodecoll&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 182245 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 182255 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 182261 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.s.IndexSchema [utilizenodecoll_shard2_replica_n9] Schema name=cloud-dynamic
[junit4] 2> 182294 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 182294 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.c.CoreContainer Creating SolrCore 'utilizenodecoll_shard2_replica_n9' using configuration from collection utilizenodecoll, trusted=true
[junit4] 2> 182294 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37541.solr.core.utilizenodecoll.shard2.replica_n9' (registry 'solr.core.utilizenodecoll.shard2.replica_n9') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@79d33b41
[junit4] 2> 182294 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 182295 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.c.SolrCore [[utilizenodecoll_shard2_replica_n9] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node5/utilizenodecoll_shard2_replica_n9], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001/tempDir-001/node5/./utilizenodecoll_shard2_replica_n9/data/]
[junit4] 2> 182337 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 182338 INFO (qtp283626442-3344) [n:127.0.0.1:37541_solr c:utilizenodecoll s:shard2 r:core_node10 x:utilizenodecoll_shard2_replica_n9] o.a.s.u.UpdateLog Initializing UpdateLog: dataDi
[...truncated too long message...]
junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:380)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:530)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:289)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:149)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
[junit4] 2> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 184261 INFO (qtp1411202021-3221) [n:127.0.0.1:33884_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:57758_solr&onlyIfLeaderActive=true&core=utilizenodecoll_shard1_replica_n1&coreNodeName=core_node12&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=400 QTime=1001
[junit4] 2> 188481 INFO (recoveryExecutor-482-thread-1-processing-n:127.0.0.1:57758_solr x:utilizenodecoll_shard1_replica_n11 s:shard1 c:utilizenodecoll r:core_node12) [n:127.0.0.1:57758_solr c:utilizenodecoll s:shard1 r:core_node12 x:utilizenodecoll_shard1_replica_n11] o.a.s.c.RecoveryStrategy RecoveryStrategy has been closed
[junit4] 2> 188481 INFO (recoveryExecutor-482-thread-1-processing-n:127.0.0.1:57758_solr x:utilizenodecoll_shard1_replica_n11 s:shard1 c:utilizenodecoll r:core_node12) [n:127.0.0.1:57758_solr c:utilizenodecoll s:shard1 r:core_node12 x:utilizenodecoll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Finished recovery process, successful=[false]
[junit4] 2> 188481 INFO (recoveryExecutor-482-thread-1-processing-n:127.0.0.1:57758_solr x:utilizenodecoll_shard1_replica_n11 s:shard1 c:utilizenodecoll r:core_node12) [n:127.0.0.1:57758_solr c:utilizenodecoll s:shard1 r:core_node12 x:utilizenodecoll_shard1_replica_n11] o.a.s.c.SolrCore [utilizenodecoll_shard1_replica_n11] CLOSING SolrCore org.apache.solr.core.SolrCore@526d5046
[junit4] 2> 188481 INFO (recoveryExecutor-482-thread-1-processing-n:127.0.0.1:57758_solr x:utilizenodecoll_shard1_replica_n11 s:shard1 c:utilizenodecoll r:core_node12) [n:127.0.0.1:57758_solr c:utilizenodecoll s:shard1 r:core_node12 x:utilizenodecoll_shard1_replica_n11] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.utilizenodecoll.shard1.replica_n11, tag=1382895686
[junit4] 2> 188481 INFO (recoveryExecutor-482-thread-1-processing-n:127.0.0.1:57758_solr x:utilizenodecoll_shard1_replica_n11 s:shard1 c:utilizenodecoll r:core_node12) [n:127.0.0.1:57758_solr c:utilizenodecoll s:shard1 r:core_node12 x:utilizenodecoll_shard1_replica_n11] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@40ecdbba: rootName = solr_57758, domain = solr.core.utilizenodecoll.shard1.replica_n11, service url = null, agent id = null] for registry solr.core.utilizenodecoll.shard1.replica_n11 / com.codahale.metrics.MetricRegistry@15c42da9
[junit4] 2> 188490 INFO (recoveryExecutor-482-thread-1-processing-n:127.0.0.1:57758_solr x:utilizenodecoll_shard1_replica_n11 s:shard1 c:utilizenodecoll r:core_node12) [n:127.0.0.1:57758_solr c:utilizenodecoll s:shard1 r:core_node12 x:utilizenodecoll_shard1_replica_n11] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.utilizenodecoll.shard1.leader, tag=1382895686
[junit4] 2> 188490 WARN (recoveryExecutor-482-thread-1-processing-n:127.0.0.1:57758_solr x:utilizenodecoll_shard1_replica_n11 s:shard1 c:utilizenodecoll r:core_node12) [n:127.0.0.1:57758_solr c:utilizenodecoll s:shard1 r:core_node12 x:utilizenodecoll_shard1_replica_n11] o.a.s.c.RecoveryStrategy Stopping recovery for core=[utilizenodecoll_shard1_replica_n11] coreNodeName=[core_node12]
[junit4] 2> 188493 INFO (jetty-closer-414-thread-6) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@438d9a83{/solr,null,UNAVAILABLE}
[junit4] 2> 188515 INFO (jetty-closer-414-thread-6) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 188516 ERROR (SUITE-TestUtilizeNode-seed#[4BDCD6082D33F991]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 188516 INFO (SUITE-TestUtilizeNode-seed#[4BDCD6082D33F991]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54329 54329
[junit4] 2> 188614 INFO (Thread-1786) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:54329 54329
[junit4] 2> 188615 WARN (Thread-1786) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 12 /solr/collections/utilizenodecoll/terms/shard2
[junit4] 2> 12 /solr/collections/utilizenodecoll/terms/shard1
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 6 /solr/security.json
[junit4] 2> 6 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 52 /solr/collections/utilizenodecoll/state.json
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 3 /solr/overseer_elect/election/73279364246798346-127.0.0.1:48720_solr-n_0000000001
[junit4] 2> 2 /solr/collections/utilizenodecoll/leader_elect/shard2/election/73279364246798346-core_node7-n_0000000000
[junit4] 2> 2 /solr/collections/utilizenodecoll/leader_elect/shard1/election/73279364246798348-core_node2-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/live_nodes
[junit4] 2> 9 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.TestUtilizeNode_4BDCD6082D33F991-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@357e603a), locale=fr-CH, timezone=US/East-Indiana
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=247115344,total=492306432
[junit4] 2> NOTE: All tests run in this JVM: [SynonymTokenizerTest, TestGroupingSearch, DistributedFacetPivotLargeTest, ImplicitSnitchTest, ActionThrottleTest, TestFieldCacheSortRandom, BlockJoinFacetSimpleTest, TestInfoStreamLogging, DistributedTermsComponentTest, TestIndexingPerformance, ReplaceNodeTest, SchemaApiFailureTest, TestReloadAndDeleteDocs, TestCollationFieldDocValues, SortSpecParsingTest, LargeFieldTest, DirectUpdateHandlerTest, MigrateRouteKeyTest, TestHighlightDedupGrouping, ConcurrentCreateRoutedAliasTest, TestSolrCloudWithDelegationTokens, UpdateParamsTest, TestRequestForwarding, TestHdfsUpdateLog, CurrencyFieldTypeTest, TestSolrQueryParser, TestConfigsApi, TestUtilizeNode]
[junit4] Completed [55/776 (1!)] on J0 in 11.13s, 1 test, 1 failure <<< FAILURES!
[...truncated 50786 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2313 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2313/
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeLostTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([5BA9FD6FC1C32DF3:5F57547DFB09AF69]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testNodeLostTrigger(TriggerIntegrationTest.java:521)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([5BA9FD6FC1C32DF3:E1A5CAE09E2BFBBC]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNull(Assert.java:551)
at org.junit.Assert.assertNull(Assert.java:562)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testMetricTrigger(TriggerIntegrationTest.java:1575)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 14068 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> 2099765 INFO (SUITE-TriggerIntegrationTest-seed#[5BA9FD6FC1C32DF3]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/init-core-data-001
[junit4] 2> 2099766 WARN (SUITE-TriggerIntegrationTest-seed#[5BA9FD6FC1C32DF3]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=8 numCloses=8
[junit4] 2> 2099766 INFO (SUITE-TriggerIntegrationTest-seed#[5BA9FD6FC1C32DF3]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2099767 INFO (SUITE-TriggerIntegrationTest-seed#[5BA9FD6FC1C32DF3]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2099768 INFO (SUITE-TriggerIntegrationTest-seed#[5BA9FD6FC1C32DF3]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001
[junit4] 2> 2099768 INFO (SUITE-TriggerIntegrationTest-seed#[5BA9FD6FC1C32DF3]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2099768 INFO (Thread-6905) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2099768 INFO (Thread-6905) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2099776 ERROR (Thread-6905) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2099868 INFO (SUITE-TriggerIntegrationTest-seed#[5BA9FD6FC1C32DF3]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:35763
[junit4] 2> 2099871 INFO (zkConnectionManagerCallback-7066-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2099875 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1045631573a0000, likely client has closed socket
[junit4] 2> 2099877 INFO (jetty-launcher-7063-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2099878 INFO (jetty-launcher-7063-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2099878 INFO (jetty-launcher-7063-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2099878 INFO (jetty-launcher-7063-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2099878 INFO (jetty-launcher-7063-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@43f89761{/solr,null,AVAILABLE}
[junit4] 2> 2099879 INFO (jetty-launcher-7063-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@23eac220{SSL,[ssl, http/1.1]}{127.0.0.1:๓๒๘๙๒}
[junit4] 2> 2099884 INFO (jetty-launcher-7063-thread-1) [ ] o.e.j.s.Server Started @๒๑๐๑๘๗๘ms
[junit4] 2> 2099884 INFO (jetty-launcher-7063-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=32892}
[junit4] 2> 2099884 ERROR (jetty-launcher-7063-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2099884 INFO (jetty-launcher-7063-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2099884 INFO (jetty-launcher-7063-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2099884 INFO (jetty-launcher-7063-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2099884 INFO (jetty-launcher-7063-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T18:33:52.805Z
[junit4] 2> 2099885 INFO (jetty-launcher-7063-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2099886 INFO (jetty-launcher-7063-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2099886 INFO (jetty-launcher-7063-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2099887 INFO (jetty-launcher-7063-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2099887 INFO (jetty-launcher-7063-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3391c809{/solr,null,AVAILABLE}
[junit4] 2> 2099888 INFO (jetty-launcher-7063-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@21963f2{SSL,[ssl, http/1.1]}{127.0.0.1:๕๒๙๙๓}
[junit4] 2> 2099888 INFO (jetty-launcher-7063-thread-2) [ ] o.e.j.s.Server Started @๒๑๐๑๘๘๗ms
[junit4] 2> 2099888 INFO (zkConnectionManagerCallback-7068-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2099888 INFO (jetty-launcher-7063-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52993}
[junit4] 2> 2099889 INFO (jetty-launcher-7063-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2099889 ERROR (jetty-launcher-7063-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2099889 INFO (jetty-launcher-7063-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2099889 INFO (jetty-launcher-7063-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2099889 INFO (jetty-launcher-7063-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2099889 INFO (jetty-launcher-7063-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T18:33:52.810Z
[junit4] 2> 2099891 INFO (zkConnectionManagerCallback-7070-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2099892 INFO (jetty-launcher-7063-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2099896 INFO (jetty-launcher-7063-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35763/solr
[junit4] 2> 2099897 INFO (zkConnectionManagerCallback-7074-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2099897 INFO (jetty-launcher-7063-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35763/solr
[junit4] 2> 2099898 INFO (zkConnectionManagerCallback-7078-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2099917 INFO (zkConnectionManagerCallback-7080-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2099918 INFO (zkConnectionManagerCallback-7082-thread-1-processing-n:127.0.0.1:52993_solr) [n:127.0.0.1:52993_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2099990 INFO (jetty-launcher-7063-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2099990 INFO (jetty-launcher-7063-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:32892_solr
[junit4] 2> 2099991 INFO (jetty-launcher-7063-thread-2) [n:127.0.0.1:52993_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2099991 INFO (jetty-launcher-7063-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.Overseer Overseer (id=73278263861575685-127.0.0.1:32892_solr-n_0000000000) starting
[junit4] 2> 2099991 INFO (jetty-launcher-7063-thread-2) [n:127.0.0.1:52993_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52993_solr
[junit4] 2> 2099993 INFO (zkCallback-7081-thread-1-processing-n:127.0.0.1:52993_solr) [n:127.0.0.1:52993_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2099994 INFO (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2099998 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 2099999 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 2100000 INFO (jetty-launcher-7063-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32892_solr
[junit4] 2> 2100000 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:52993_solr]
[junit4] 2> 2100000 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 2100000 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 2100005 INFO (zkCallback-7081-thread-1-processing-n:127.0.0.1:52993_solr) [n:127.0.0.1:52993_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2100006 INFO (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2100006 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2100006 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 2100007 DEBUG (ScheduledTrigger-8572-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2100054 INFO (jetty-launcher-7063-thread-2) [n:127.0.0.1:52993_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52993.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2100062 INFO (jetty-launcher-7063-thread-2) [n:127.0.0.1:52993_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52993.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2100062 INFO (jetty-launcher-7063-thread-2) [n:127.0.0.1:52993_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52993.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2100063 INFO (jetty-launcher-7063-thread-2) [n:127.0.0.1:52993_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001/node2/.
[junit4] 2> 2100065 INFO (jetty-launcher-7063-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32892.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2100072 INFO (jetty-launcher-7063-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32892.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2100073 INFO (jetty-launcher-7063-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32892.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2100074 INFO (jetty-launcher-7063-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001/node1/.
[junit4] 2> 2100121 INFO (zkConnectionManagerCallback-7088-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2100124 INFO (zkConnectionManagerCallback-7092-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2100124 INFO (SUITE-TriggerIntegrationTest-seed#[5BA9FD6FC1C32DF3]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2100125 INFO (SUITE-TriggerIntegrationTest-seed#[5BA9FD6FC1C32DF3]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35763/solr ready
[junit4] 2> 2100141 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testListeners
[junit4] 2> 2100142 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 2100142 DEBUG (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 2100142 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 2100144 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2100144 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2100144 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 2100171 INFO (qtp602428110-27194) [n:127.0.0.1:32892_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2100187 INFO (qtp798043386-27200) [n:127.0.0.1:52993_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2100187 DEBUG (qtp602428110-27193) [n:127.0.0.1:32892_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2100187 INFO (qtp602428110-27193) [n:127.0.0.1:32892_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 2100188 DEBUG (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 2100188 DEBUG (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:32892_solr, 127.0.0.1:52993_solr]
[junit4] 2> 2100188 DEBUG (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 2100188 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 2100189 INFO (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2100190 DEBUG (ScheduledTrigger-8572-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2100190 DEBUG (ScheduledTrigger-8572-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 2100191 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2100191 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 2100194 INFO (qtp602428110-27191) [n:127.0.0.1:32892_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2100195 INFO (qtp798043386-27203) [n:127.0.0.1:52993_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2100195 DEBUG (qtp798043386-27202) [n:127.0.0.1:52993_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2100196 INFO (qtp798043386-27202) [n:127.0.0.1:52993_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 2100196 DEBUG (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 2100196 DEBUG (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:32892_solr, 127.0.0.1:52993_solr]
[junit4] 2> 2100196 DEBUG (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 2100196 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 2100197 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2100198 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 2100198 INFO (qtp602428110-27193) [n:127.0.0.1:32892_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2100199 INFO (qtp798043386-27206) [n:127.0.0.1:52993_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2100200 DEBUG (qtp798043386-27200) [n:127.0.0.1:52993_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2100227 INFO (qtp798043386-27200) [n:127.0.0.1:52993_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=30
[junit4] 2> 2100227 DEBUG (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 2100228 DEBUG (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:32892_solr, 127.0.0.1:52993_solr]
[junit4] 2> 2100228 DEBUG (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 2100228 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 2100228 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2100228 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 2100229 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2100229 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2100230 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2100230 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2100230 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b98ab5a{/solr,null,AVAILABLE}
[junit4] 2> 2100230 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@74be4def{SSL,[ssl, http/1.1]}{127.0.0.1:๕๒๙๑๔}
[junit4] 2> 2100231 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.Server Started @๒๑๐๒๒๓๐ms
[junit4] 2> 2100231 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52914}
[junit4] 2> 2100231 ERROR (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2100231 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2100231 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2100232 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2100232 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T18:33:53.153Z
[junit4] 2> 2100233 INFO (zkConnectionManagerCallback-7094-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2100233 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2100240 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35763/solr
[junit4] 2> 2100241 INFO (zkConnectionManagerCallback-7098-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2100242 INFO (zkConnectionManagerCallback-7100-thread-1-processing-n:127.0.0.1:52914_solr) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2100248 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2100249 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:52914_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2100251 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:52914_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52914_solr
[junit4] 2> 2100252 INFO (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2100252 INFO (zkCallback-7081-thread-1-processing-n:127.0.0.1:52993_solr) [n:127.0.0.1:52993_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2100253 INFO (zkCallback-7099-thread-1-processing-n:127.0.0.1:52914_solr) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2100254 INFO (zkCallback-7091-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2100335 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:52914_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52914.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2100349 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:52914_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52914.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2100349 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:52914_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52914.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2100351 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:52914_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001/node3/.
[junit4] 2> 2101192 DEBUG (ScheduledTrigger-8572-thread-2) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2101192 DEBUG (ScheduledTrigger-8572-thread-2) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2101192 DEBUG (ScheduledTrigger-8572-thread-2) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:52914_solr at time 18625944776954450
[junit4] 2> 2102193 DEBUG (ScheduledTrigger-8572-thread-2) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2102193 DEBUG (ScheduledTrigger-8572-thread-2) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2102194 DEBUG (ScheduledTrigger-8572-thread-2) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL firing registered processor for nodes: [127.0.0.1:52914_solr] added at times [18625944776954450], now=18625945778109845
[junit4] 2> 2102194 DEBUG (ScheduledTrigger-8572-thread-2) [n:127.0.0.1:32892_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"422c32b6095252Td6fjfbmkyvylbg1d36ibh8hi",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18625944776954450,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18625944776954450],
[junit4] 2> "nodeNames":["127.0.0.1:52914_solr"]}}
[junit4] 2> 2102195 DEBUG (ScheduledTrigger-8572-thread-2) [n:127.0.0.1:32892_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_triggerL]
[junit4] 2> 2102196 INFO (ScheduledTrigger-8572-thread-2) [n:127.0.0.1:32892_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2102197 DEBUG (AutoscalingActionExecutor-8573-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"422c32b6095252Td6fjfbmkyvylbg1d36ibh8hi",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18625944776954450,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18625944776954450],
[junit4] 2> "_enqueue_time_":18625945779450074,
[junit4] 2> "nodeNames":["127.0.0.1:52914_solr"]}}
[junit4] 2> 2102200 DEBUG (AutoscalingActionExecutor-8573-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerL after 5000ms
[junit4] 2> 2102200 DEBUG (AutoscalingActionExecutor-8573-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 3 ms for event id=422c32b6095252Td6fjfbmkyvylbg1d36ibh8hi
[junit4] 2> 2104201 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2104202 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2104202 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2104202 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2104203 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3468219{/solr,null,AVAILABLE}
[junit4] 2> 2104203 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7e69c38e{SSL,[ssl, http/1.1]}{127.0.0.1:๕๙๕๔๙}
[junit4] 2> 2104203 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.Server Started @๒๑๐๖๒๐๒ms
[junit4] 2> 2104203 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59549}
[junit4] 2> 2104204 ERROR (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2104204 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2104204 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2104204 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2104204 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T18:33:57.125Z
[junit4] 2> 2104206 INFO (zkConnectionManagerCallback-7104-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2104206 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2104242 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35763/solr
[junit4] 2> 2104243 INFO (zkConnectionManagerCallback-7108-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2104245 INFO (zkConnectionManagerCallback-7110-thread-1-processing-n:127.0.0.1:59549_solr) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2104251 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2104253 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:59549_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2104255 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:59549_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59549_solr
[junit4] 2> 2104256 INFO (zkCallback-7091-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2104256 INFO (zkCallback-7099-thread-1-processing-n:127.0.0.1:52914_solr) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2104256 INFO (zkCallback-7081-thread-1-processing-n:127.0.0.1:52993_solr) [n:127.0.0.1:52993_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2104256 INFO (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2104264 INFO (zkCallback-7109-thread-1-processing-n:127.0.0.1:59549_solr) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2104315 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:59549_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59549.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2104324 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:59549_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59549.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2104325 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:59549_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59549.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2104326 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:59549_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001/node4/.
[junit4] 2> 2107201 DEBUG (ScheduledTrigger-8572-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2107201 DEBUG (ScheduledTrigger-8572-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2107201 DEBUG (ScheduledTrigger-8572-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:59549_solr at time 18625950785209875
[junit4] 2> 2108202 DEBUG (ScheduledTrigger-8572-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 2108202 DEBUG (ScheduledTrigger-8572-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2108202 DEBUG (ScheduledTrigger-8572-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL firing registered processor for nodes: [127.0.0.1:59549_solr] added at times [18625950785209875], now=18625951786271301
[junit4] 2> 2108202 DEBUG (ScheduledTrigger-8572-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"422c341c280613Td6fjfbmkyvylbg1d36ibh8hk",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18625950785209875,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18625950785209875],
[junit4] 2> "nodeNames":["127.0.0.1:59549_solr"]}}
[junit4] 2> 2108202 DEBUG (ScheduledTrigger-8572-thread-1) [n:127.0.0.1:32892_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_triggerL]
[junit4] 2> 2108202 DEBUG (AutoscalingActionExecutor-8573-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"422c341c280613Td6fjfbmkyvylbg1d36ibh8hk",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18625950785209875,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18625950785209875],
[junit4] 2> "_enqueue_time_":18625951786523609,
[junit4] 2> "nodeNames":["127.0.0.1:59549_solr"]}}
[junit4] 2> 2108203 WARN (AutoscalingActionExecutor-8573-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> java.lang.Exception: Error executing action: test1 for trigger event: {
[junit4] 2> "id":"422c341c280613Td6fjfbmkyvylbg1d36ibh8hk",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18625950785209875,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18625950785209875],
[junit4] 2> "_enqueue_time_":18625951786523609,
[junit4] 2> "nodeNames":["127.0.0.1:59549_solr"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:302)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.RuntimeException: failure
[junit4] 2> at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction.process(TriggerIntegrationTest.java:989)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:299)
[junit4] 2> ... 6 more
[junit4] 2> 2108203 DEBUG (AutoscalingActionExecutor-8573-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerL after 5000ms
[junit4] 2> 2108203 DEBUG (AutoscalingActionExecutor-8573-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 0 ms for event id=422c341c280613Td6fjfbmkyvylbg1d36ibh8hk
[junit4] 2> 2110203 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testListeners
[junit4] 2> 2110234 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
[junit4] 2> 2110235 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 6
[junit4] 2> 2110235 DEBUG (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 2110235 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 2110236 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2110236 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2110237 DEBUG (OverseerAutoScalingTriggerThread-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 2110240 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2110241 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2110241 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2110241 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2110242 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@65b7d11c{/solr,null,AVAILABLE}
[junit4] 2> 2110242 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2c9f6ecf{SSL,[ssl, http/1.1]}{127.0.0.1:๓๓๘๔๘}
[junit4] 2> 2110243 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.Server Started @๒๑๑๒๒๔๒ms
[junit4] 2> 2110243 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33848}
[junit4] 2> 2110243 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2110243 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2110243 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2110243 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2110243 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T18:34:03.164Z
[junit4] 2> 2110245 INFO (zkConnectionManagerCallback-7114-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2110245 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2110252 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35763/solr
[junit4] 2> 2110253 INFO (zkConnectionManagerCallback-7118-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2110255 INFO (zkConnectionManagerCallback-7120-thread-1-processing-n:127.0.0.1:33848_solr) [n:127.0.0.1:33848_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2110259 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33848_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2110261 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33848_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2110263 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33848_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33848_solr
[junit4] 2> 2110264 INFO (zkCallback-7099-thread-1-processing-n:127.0.0.1:52914_solr) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2110264 INFO (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2110264 INFO (zkCallback-7109-thread-1-processing-n:127.0.0.1:59549_solr) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2110264 INFO (zkCallback-7081-thread-1-processing-n:127.0.0.1:52993_solr) [n:127.0.0.1:52993_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2110264 INFO (zkCallback-7091-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2110265 INFO (zkCallback-7119-thread-1-processing-n:127.0.0.1:33848_solr) [n:127.0.0.1:33848_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2110391 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33848_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33848.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2110399 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33848_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33848.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2110400 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33848_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33848.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2110401 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33848_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001/node5/.
[junit4] 2> 2110452 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2110453 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2110453 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2110453 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2110453 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@633ed9f5{/solr,null,AVAILABLE}
[junit4] 2> 2110454 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@37647f87{SSL,[ssl, http/1.1]}{127.0.0.1:๔๔๖๖๓}
[junit4] 2> 2110454 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.Server Started @๒๑๑๒๔๕๓ms
[junit4] 2> 2110454 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44663}
[junit4] 2> 2110454 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2110454 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2110454 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2110454 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2110454 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T18:34:03.375Z
[junit4] 2> 2110456 INFO (zkConnectionManagerCallback-7124-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2110456 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2110463 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35763/solr
[junit4] 2> 2110464 INFO (zkConnectionManagerCallback-7128-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2110465 INFO (zkConnectionManagerCallback-7130-thread-1-processing-n:127.0.0.1:44663_solr) [n:127.0.0.1:44663_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2110471 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:44663_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 2110472 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:44663_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2110474 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:44663_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44663_solr
[junit4] 2> 2110475 INFO (zkCallback-7081-thread-1-processing-n:127.0.0.1:52993_solr) [n:127.0.0.1:52993_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2110475 INFO (zkCallback-7109-thread-1-processing-n:127.0.0.1:59549_solr) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2110475 INFO (zkCallback-7099-thread-1-processing-n:127.0.0.1:52914_solr) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2110475 INFO (zkCallback-7091-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2110475 INFO (zkCallback-7119-thread-1-processing-n:127.0.0.1:33848_solr) [n:127.0.0.1:33848_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2110475 INFO (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2110477 INFO (zkCallback-7129-thread-1-processing-n:127.0.0.1:44663_solr) [n:127.0.0.1:44663_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 2110579 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:44663_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44663.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2110589 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:44663_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44663.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2110589 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:44663_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44663.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2110591 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:44663_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001/node6/.
[junit4] 2> 2110673 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T00:27:37+03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2110674 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2110674 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2110674 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2110674 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3f7123dc{/solr,null,AVAILABLE}
[junit4] 2> 2110674 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@33ad7657{SSL,[ssl, http/1.1]}{127.0.0.1:๓๓๐๘๙}
[junit4] 2> 2110675 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.e.j.s.Server Started @๒๑๑๒๖๗๔ms
[junit4] 2> 2110675 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33089}
[junit4] 2> 2110675 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2110675 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2110675 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2110675 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2110675 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T18:34:03.596Z
[junit4] 2> 2110677 INFO (zkConnectionManagerCallback-7134-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2110677 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2110684 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35763/solr
[junit4] 2> 2110685 INFO (zkConnectionManagerCallback-7138-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2110687 INFO (zkConnectionManagerCallback-7140-thread-1-processing-n:127.0.0.1:33089_solr) [n:127.0.0.1:33089_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2110693 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33089_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
[junit4] 2> 2110695 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33089_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2110697 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33089_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33089_solr
[junit4] 2> 2110698 INFO (zkCallback-7129-thread-1-processing-n:127.0.0.1:44663_solr) [n:127.0.0.1:44663_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2110698 INFO (zkCallback-7109-thread-1-processing-n:127.0.0.1:59549_solr) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2110698 INFO (zkCallback-7081-thread-1-processing-n:127.0.0.1:52993_solr) [n:127.0.0.1:52993_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2110703 INFO (zkCallback-7119-thread-1-processing-n:127.0.0.1:33848_solr) [n:127.0.0.1:33848_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2110703 INFO (zkCallback-7079-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2110703 INFO (zkCallback-7099-thread-1-processing-n:127.0.0.1:52914_solr) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2110703 INFO (zkCallback-7091-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2110704 INFO (zkCallback-7139-thread-1-processing-n:127.0.0.1:33089_solr) [n:127.0.0.1:33089_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (6) -> (7)
[junit4] 2> 2110757 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33089_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33089.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2110767 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33089_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33089.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2110767 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33089_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33089.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2110768 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[5BA9FD6FC1C32DF3]) [n:127.0.0.1:33089_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001/node7/.
[junit4] 2> 2110856 INFO (qtp602428110-27191) [n:127.0.0.1:32892_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=collection1&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2110859 INFO (OverseerThreadFactory-8574-thread-1-processing-n:127.0.0.1:32892_solr) [n:127.0.0.1:32892_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 2110968 INFO (OverseerStateUpdate-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:52914/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2110971 INFO (OverseerStateUpdate-73278263861575685-127.0.0.1:32892_solr-n_0000000000) [n:127.0.0.1:32892_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:59549/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2111199 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n2&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2111199 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2111203 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2111203 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2111306 INFO (zkCallback-7099-thread-1-processing-n:127.0.0.1:52914_solr) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2111306 INFO (zkCallback-7109-thread-1-processing-n:127.0.0.1:59549_solr) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2111306 INFO (zkCallback-7109-thread-2-processing-n:127.0.0.1:59549_solr) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2111306 INFO (zkCallback-7099-thread-2-processing-n:127.0.0.1:52914_solr) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2112210 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2112214 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2112217 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema [collection1_shard1_replica_n2] Schema name=minimal
[junit4] 2> 2112231 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2112231 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n2' using configuration from collection collection1, trusted=true
[junit4] 2> 2112232 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59549.solr.core.collection1.shard1.replica_n2' (registry 'solr.core.collection1.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2112232 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2112232 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2112232 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore [[collection1_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001/node4/collection1_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001/node4/./collection1_shard1_replica_n2/data/]
[junit4] 2> 2112234 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2112234 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from collection collection1, trusted=true
[junit4] 2> 2112234 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52914.solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@3aaae170
[junit4] 2> 2112234 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2112234 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001/node3/collection1_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001/tempDir-001/node3/./collection1_shard1_replica_n1/data/]
[junit4] 2> 2112274 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2112274 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2112274 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2112280 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2112281 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2112281 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2112282 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2112282 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2112282 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@703822aa[collection1_shard1_replica_n2] main]
[junit4] 2> 2112283 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2112283 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2112284 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6b43e903[collection1_shard1_replica_n1] main]
[junit4] 2> 2112284 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2112285 INFO (searcherExecutor-8609-thread-1-processing-n:127.0.0.1:59549_solr x:collection1_shard1_replica_n2 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore [collection1_shard1_replica_n2] Registered new searcher Searcher@703822aa[collection1_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2112285 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592130582825926656
[junit4] 2> 2112285 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2112285 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2112286 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2112287 INFO (searcherExecutor-8610-thread-1-processing-n:127.0.0.1:52914_solr x:collection1_shard1_replica_n1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [collection1_shard1_replica_n1] Registered new searcher Searcher@6b43e903[collection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2112287 INFO (qtp1118394504-27255) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592130582828023808
[junit4] 2> 2112292 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2112393 INFO (zkCallback-7109-thread-1-processing-n:127.0.0.1:59549_solr) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2112393 INFO (zkCallback-7099-thread-2-processing-n:127.0.0.1:52914_solr) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2112393 INFO (zkCallback-7099-thread-1-processing-n:127.0.0.1:52914_solr) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2112393 INFO (zkCallback-7109-thread-2-processing-n:127.0.0.1:59549_solr) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2112792 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2112793 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2112793 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:59549/solr/collection1_shard1_replica_n2/
[junit4] 2> 2112793 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n2 url=https://127.0.0.1:59549/solr START replicas=[https://127.0.0.1:52914/solr/collection1_shard1_replica_n1/] nUpdates=100
[junit4] 2> 2112798 INFO (qtp1118394504-27259) [n:127.0.0.1:52914_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.S.Request [collection1_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2113094 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n2 url=https://127.0.0.1:59549/solr DONE. We have no versions. sync failed.
[junit4] 2> 2113094 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2113094 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2113094 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2113098 INFO (qtp904791036-27283) [n:127.0.0.1:59549_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:59549/solr/collection1_shard1_replica_n2/ shard1
[junit4] 2> 2113200 INFO (zkCallback-7099-thread-2-processing-n:127.0.0.1:52914_solr) [n:127.0.0.1:52914_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2113200 INFO (zkCallback-7109-thread-2-processing-n:127.0.0.1:59549_solr) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [7])
[junit4] 2> 2113200 INFO (zkCallback-7109-thread-1-processing-n:127.0.0.1:59549_solr) [n:127.0.0.1:59549_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataCh
[...truncated too long message...]
44663_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 2296456 DEBUG (jetty-closer-7064-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 2296472 INFO (OverseerStateUpdate-73278263861575700-127.0.0.1:44663_solr-n_0000000005) [n:127.0.0.1:44663_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44663_solr
[junit4] 2> 2296483 INFO (jetty-closer-7064-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@633ed9f5{/solr,null,UNAVAILABLE}
[junit4] 2> 2296484 INFO (jetty-closer-7064-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2297838 WARN (zkCallback-7287-thread-3-processing-n:127.0.0.1:32910_solr) [n:127.0.0.1:32910_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2297838 INFO (jetty-closer-7064-thread-13) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5e66dc7c{/solr,null,UNAVAILABLE}
[junit4] 2> 2297838 INFO (jetty-closer-7064-thread-13) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2297871 INFO (jetty-closer-7064-thread-11) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6e5c9cf4{/solr,null,UNAVAILABLE}
[junit4] 2> 2297871 INFO (jetty-closer-7064-thread-11) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2297872 ERROR (SUITE-TriggerIntegrationTest-seed#[5BA9FD6FC1C32DF3]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2297872 INFO (SUITE-TriggerIntegrationTest-seed#[5BA9FD6FC1C32DF3]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35763 35763
[junit4] 2> 2297980 INFO (Thread-6905) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35763 35763
[junit4] 2> 2297981 WARN (Thread-6905) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 38 /solr/collections/collection1/terms/shard1
[junit4] 2> 22 /solr/aliases.json
[junit4] 2> 21 /solr/security.json
[junit4] 2> 7 /solr/configs/conf
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 121 /solr/collections/collection1/state.json
[junit4] 2> 22 /solr/clusterprops.json
[junit4] 2> 22 /solr/clusterstate.json
[junit4] 2> 13 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 4 /solr/overseer_elect/election/73278263861575700-127.0.0.1:44663_solr-n_0000000005
[junit4] 2> 3 /solr/overseer_elect/election/73278263861575737-127.0.0.1:46370_solr-n_0000000017
[junit4] 2> 3 /solr/collections/collection1/leader_elect/shard1/election/73278263861575700-core_node8-n_0000000003
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73278263861575694-core_node4-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/73278263861575719-127.0.0.1:36094_solr-n_0000000011
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73278263861575691-core_node3-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/73278263861575713-127.0.0.1:38107_solr-n_0000000009
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/live_nodes
[junit4] 2> 22 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/queue
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_5BA9FD6FC1C32DF3-001
[junit4] 2> Feb 11, 2018 6:37:10 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=FST50, x_s=PostingsFormat(name=Direct)}, docValues:{}, maxPointsInLeafNode=144, maxMBSortInHeap=6.471241412829087, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3c9e464b), locale=th-TH-u-nu-thai-x-lvariant-TH, timezone=America/Knox_IN
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=162191424,total=481820672
[junit4] 2> NOTE: All tests run in this JVM: [DistribDocExpirationUpdateProcessorTest, AtomicUpdateProcessorFactoryTest, DeleteShardTest, TestFieldCacheSortRandom, LargeFieldTest, TestNoOpRegenerator, TestFieldCache, DistributedFacetPivotSmallTest, TestMaxTokenLenTokenizer, HDFSCollectionsAPITest, TestLegacyFieldCache, TestExportWriter, TestNumericRangeQuery64, TestReqParamsAPI, HdfsChaosMonkeyNothingIsSafeTest, TestRandomFaceting, JvmMetricsTest, SpatialHeatmapFacetsTest, ConfigureRecoveryStrategyTest, RAMDirectoryFactoryTest, HighlighterConfigTest, ZkSolrClientTest, TestStressCloudBlindAtomicUpdates, DefaultValueUpdateProcessorTest, OverseerCollectionConfigSetProcessorTest, TestSolrFieldCacheBean, DistributedFacetSimpleRefinementLongTailTest, UUIDFieldTest, DistributedQueryComponentOptimizationTest, TestLeaderElectionWithEmptyReplica, SuggesterWFSTTest, TestBulkSchemaConcurrent, LeaderFailureAfterFreshStartTest, BJQParserTest, ActionThrottleTest, TestSolrCloudWithDelegationTokens, TestPayloadScoreQParserPlugin, TestPHPSerializedResponseWriter, TestDistributedStatsComponentCardinality, InfixSuggestersTest, AutoScalingHandlerTest, NodeAddedTriggerTest, TestComplexPhraseLeadingWildcard, SynonymTokenizerTest, TestFileDictionaryLookup, TestReloadAndDeleteDocs, TestDocumentBuilder, SmileWriterTest, TestConfigSetsAPIZkFailure, TestWriterPerf, CursorPagingTest, TestTrieFacet, UninvertDocValuesMergePolicyTest, ChaosMonkeySafeLeaderWithPullReplicasTest, CoreMergeIndexesAdminHandlerTest, XmlUpdateRequestHandlerTest, BadCopyFieldTest, HLLSerializationTest, TestRequestStatusCollectionAPI, TermVectorComponentDistributedTest, HdfsUnloadDistributedZkTest, StatsComponentTest, TestBackupRepositoryFactory, TestWordDelimiterFilterFactory, TestCloudPseudoReturnFields, TestConfig, TestLegacyTerms, SearchHandlerTest, TestPrepRecovery, ClassificationUpdateProcessorIntegrationTest, HdfsSyncSliceTest, TestGroupingSearch, TestHighlightDedupGrouping, MigrateRouteKeyTest, CustomCollectionTest, ExitableDirectoryReaderTest, OverseerStatusTest, TestLMDirichletSimilarityFactory, TestRuleBasedAuthorizationPlugin, SortSpecParsingTest, ImplicitSnitchTest, CreateCollectionCleanupTest, TestLRUStatsCache, SimpleFacetsTest, PluginInfoTest, SystemLogListenerTest, TestNodeAddedTrigger, RecoveryAfterSoftCommitTest, AutoscalingHistoryHandlerTest, CdcrReplicationHandlerTest, SolrRequestParserTest, TestOmitPositions, TestPseudoReturnFields, TestJavabinTupleStreamParser, RequestHandlersTest, TestIndexingPerformance, TestCSVLoader, OpenExchangeRatesOrgProviderTest, SolrShardReporterTest, OutputWriterTest, DirectUpdateHandlerTest, TestDistributedSearch, SharedFSAutoReplicaFailoverTest, TestCollationFieldDocValues, HighlighterMaxOffsetTest, SolrGangliaReporterTest, TestClusterStateMutator, SolrCoreCheckLockOnStartupTest, TestEmbeddedSolrServerSchemaAPI, TestPhraseSuggestions, TestInfoStreamLogging, DeleteLastCustomShardedReplicaTest, TestSchemaNameResource, TestSolrDeletionPolicy1, TestMergePolicyConfig, TestCoreBackup, TestClassicSimilarityFactory, SolrIndexConfigTest, DistributedDebugComponentTest, TestTlogReplica, VMParamsZkACLAndCredentialsProvidersTest, TestMinMaxOnMultiValuedField, UUIDUpdateProcessorFallbackTest, TestSolrCLIRunExample, TestHashPartitioner, DistributedQueryElevationComponentTest, BasicDistributedZkTest, OpenCloseCoreStressTest, ShardRoutingTest, BasicZkTest, FullSolrCloudDistribCmdsTest, TestZkChroot, TestRandomDVFaceting, ZkCLITest, ShardRoutingCustomTest, DistributedSpellCheckComponentTest, ZkControllerTest, TestStressReorder, TestReload, DistributedTermsComponentTest, TestCoreContainer, TestSolr4Spatial, SpellCheckComponentTest, TestFiltering, TestBadConfig, TestIndexSearcher, SoftAutoCommitTest, PreAnalyzedFieldTest, PrimUtilsTest, SpellingQueryConverterTest, DOMUtilTest, TestSolrJ, TestLRUCache, TestUtils, ZkNodePropsTest, SliceStateTest, CircularListTest, DistributedMLTComponentTest, DistributedIntervalFacetingTest, SolrTestCaseJ4Test, TestCrossCoreJoin, TestDistributedMissingSort, TestLuceneIndexBackCompat, TestJettySolrRunner, ConnectionReuseTest, AssignBackwardCompatibilityTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, DeleteInactiveReplicaTest, HealthCheckHandlerTest, HttpPartitionOnCommitTest, LeaderFailoverAfterPartitionTest, MoveReplicaHDFSTest, TestDeleteCollectionOnDownNodes, TestLeaderInitiatedRecoveryThread, TestOnReconnectListenerSupport, TestRandomRequestDistribution, TestStressLiveNodes, TestUtilizeNode, ZkShardTermsTest, CollectionsAPIAsyncDistributedZkTest, HdfsCollectionsAPIDistributedZkTest, SimpleCollectionCreateDeleteTest, TestCollectionsAPIViaSolrCloudCluster, TestHdfsCloudBackupRestore, TestReplicaProperties, ExecutePlanActionTest, NodeLostTriggerTest, SearchRateTriggerTest, TriggerIntegrationTest]
[junit4] Completed [630/776 (1!)] on J0 in 198.82s, 14 tests, 2 failures <<< FAILURES!
[...truncated 48791 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2312 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2312/
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testListeners
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([2136A2C8734F856E:CCF635D8CE2318B7]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testListeners(TriggerIntegrationTest.java:1053)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication
Error Message:
Index: 0, Size: 0
Stack Trace:
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at __randomizedtesting.SeedInfo.seed([2136A2C8734F856E:357EF99D50483870]:0)
at java.util.ArrayList.rangeCheck(ArrayList.java:653)
at java.util.ArrayList.get(ArrayList.java:429)
at org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication(TestReplicationHandler.java:561)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12744 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_2136A2C8734F856E-001/init-core-data-001
[junit4] 2> 651519 WARN (SUITE-TriggerIntegrationTest-seed#[2136A2C8734F856E]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=15 numCloses=15
[junit4] 2> 651519 INFO (SUITE-TriggerIntegrationTest-seed#[2136A2C8734F856E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 651520 INFO (SUITE-TriggerIntegrationTest-seed#[2136A2C8734F856E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 651521 INFO (SUITE-TriggerIntegrationTest-seed#[2136A2C8734F856E]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 651521 INFO (SUITE-TriggerIntegrationTest-seed#[2136A2C8734F856E]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_2136A2C8734F856E-001/tempDir-001
[junit4] 2> 651521 INFO (SUITE-TriggerIntegrationTest-seed#[2136A2C8734F856E]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 651522 INFO (Thread-2957) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 651522 INFO (Thread-2957) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 651524 ERROR (Thread-2957) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 651622 INFO (SUITE-TriggerIntegrationTest-seed#[2136A2C8734F856E]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:56871
[junit4] 2> 651624 INFO (zkConnectionManagerCallback-4139-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 651630 INFO (jetty-launcher-4136-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 651635 INFO (jetty-launcher-4136-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 651636 INFO (jetty-launcher-4136-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 651636 INFO (jetty-launcher-4136-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 651636 INFO (jetty-launcher-4136-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 651636 INFO (jetty-launcher-4136-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@67e51850{/solr,null,AVAILABLE}
[junit4] 2> 651640 INFO (jetty-launcher-4136-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 651640 INFO (jetty-launcher-4136-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 651640 INFO (jetty-launcher-4136-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 651641 INFO (jetty-launcher-4136-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@522992b5{/solr,null,AVAILABLE}
[junit4] 2> 651641 INFO (jetty-launcher-4136-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@646b5fa0{SSL,[ssl, http/1.1]}{127.0.0.1:56690}
[junit4] 2> 651642 INFO (jetty-launcher-4136-thread-1) [ ] o.e.j.s.Server Started @654189ms
[junit4] 2> 651642 INFO (jetty-launcher-4136-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56690}
[junit4] 2> 651642 ERROR (jetty-launcher-4136-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 651642 INFO (jetty-launcher-4136-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 651642 INFO (jetty-launcher-4136-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 651642 INFO (jetty-launcher-4136-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 651642 INFO (jetty-launcher-4136-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T16:38:00.511Z
[junit4] 2> 651643 INFO (jetty-launcher-4136-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2dc06888{SSL,[ssl, http/1.1]}{127.0.0.1:46101}
[junit4] 2> 651643 INFO (jetty-launcher-4136-thread-2) [ ] o.e.j.s.Server Started @654191ms
[junit4] 2> 651643 INFO (jetty-launcher-4136-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46101}
[junit4] 2> 651643 ERROR (jetty-launcher-4136-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 651643 INFO (zkConnectionManagerCallback-4141-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 651643 INFO (jetty-launcher-4136-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 651643 INFO (jetty-launcher-4136-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 651643 INFO (jetty-launcher-4136-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 651643 INFO (jetty-launcher-4136-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T16:38:00.512Z
[junit4] 2> 651644 INFO (jetty-launcher-4136-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 651644 INFO (zkConnectionManagerCallback-4143-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 652331 INFO (jetty-launcher-4136-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 652336 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10455c741cf0001, likely client has closed socket
[junit4] 2> 652341 INFO (jetty-launcher-4136-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56871/solr
[junit4] 2> 652341 INFO (jetty-launcher-4136-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56871/solr
[junit4] 2> 652344 INFO (zkConnectionManagerCallback-4149-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 652344 INFO (zkConnectionManagerCallback-4151-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 652346 INFO (zkConnectionManagerCallback-4155-thread-1-processing-n:127.0.0.1:46101_solr) [n:127.0.0.1:46101_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 652346 INFO (zkConnectionManagerCallback-4154-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 652455 INFO (jetty-launcher-4136-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 652455 INFO (jetty-launcher-4136-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:56690_solr
[junit4] 2> 652456 INFO (jetty-launcher-4136-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.Overseer Overseer (id=73277808235708422-127.0.0.1:56690_solr-n_0000000000) starting
[junit4] 2> 652457 INFO (jetty-launcher-4136-thread-2) [n:127.0.0.1:46101_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 652458 INFO (jetty-launcher-4136-thread-2) [n:127.0.0.1:46101_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46101_solr
[junit4] 2> 652466 INFO (zkCallback-4153-thread-1-processing-n:127.0.0.1:46101_solr) [n:127.0.0.1:46101_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 652477 INFO (zkCallback-4152-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 652513 INFO (jetty-launcher-4136-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56690_solr
[junit4] 2> 652513 INFO (zkCallback-4153-thread-1-processing-n:127.0.0.1:46101_solr) [n:127.0.0.1:46101_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 652516 INFO (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 652528 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 652531 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 652531 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:56690_solr, 127.0.0.1:46101_solr]
[junit4] 2> 652531 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 652531 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 652544 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 652544 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 652548 DEBUG (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 652614 INFO (jetty-launcher-4136-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56690.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 652615 INFO (jetty-launcher-4136-thread-2) [n:127.0.0.1:46101_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46101.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 652650 INFO (jetty-launcher-4136-thread-2) [n:127.0.0.1:46101_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46101.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 652650 INFO (jetty-launcher-4136-thread-2) [n:127.0.0.1:46101_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46101.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 652651 INFO (jetty-launcher-4136-thread-2) [n:127.0.0.1:46101_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_2136A2C8734F856E-001/tempDir-001/node2/.
[junit4] 2> 652653 INFO (jetty-launcher-4136-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56690.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 652653 INFO (jetty-launcher-4136-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56690.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 652656 INFO (jetty-launcher-4136-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_2136A2C8734F856E-001/tempDir-001/node1/.
[junit4] 2> 652694 INFO (zkConnectionManagerCallback-4161-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 652696 INFO (zkConnectionManagerCallback-4165-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 652697 INFO (SUITE-TriggerIntegrationTest-seed#[2136A2C8734F856E]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 652697 INFO (SUITE-TriggerIntegrationTest-seed#[2136A2C8734F856E]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56871/solr ready
[junit4] 2> 652722 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTriggerRestoreState
[junit4] 2> 652722 DEBUG (zkCallback-4152-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 652723 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 652723 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 652727 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 652727 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 652728 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 652729 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 652730 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 652730 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 652730 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 652730 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@294bb8db{/solr,null,AVAILABLE}
[junit4] 2> 652731 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2d52a46e{SSL,[ssl, http/1.1]}{127.0.0.1:38382}
[junit4] 2> 652731 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.e.j.s.Server Started @655278ms
[junit4] 2> 652731 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38382}
[junit4] 2> 652731 ERROR (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 652731 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 652731 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 652731 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 652731 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T16:38:01.600Z
[junit4] 2> 652734 INFO (zkConnectionManagerCallback-4167-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 652735 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 652741 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56871/solr
[junit4] 2> 652742 INFO (zkConnectionManagerCallback-4171-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 652744 INFO (zkConnectionManagerCallback-4173-thread-1-processing-n:127.0.0.1:38382_solr) [n:127.0.0.1:38382_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 652748 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [n:127.0.0.1:38382_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 652749 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [n:127.0.0.1:38382_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 652751 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [n:127.0.0.1:38382_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38382_solr
[junit4] 2> 652756 INFO (zkCallback-4153-thread-1-processing-n:127.0.0.1:46101_solr) [n:127.0.0.1:46101_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 652756 INFO (zkCallback-4152-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 652756 INFO (zkCallback-4164-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 652757 INFO (zkCallback-4172-thread-1-processing-n:127.0.0.1:38382_solr) [n:127.0.0.1:38382_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 652809 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [n:127.0.0.1:38382_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38382.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 652819 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [n:127.0.0.1:38382_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38382.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 652819 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [n:127.0.0.1:38382_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38382.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 652820 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [n:127.0.0.1:38382_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_2136A2C8734F856E-001/tempDir-001/node3/.
[junit4] 2> 652916 INFO (qtp1385507505-12870) [n:127.0.0.1:38382_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 652935 INFO (qtp1419071156-12818) [n:127.0.0.1:46101_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 652938 INFO (qtp1500030909-12807) [n:127.0.0.1:56690_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 652938 DEBUG (qtp1500030909-12808) [n:127.0.0.1:56690_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 652939 INFO (qtp1500030909-12808) [n:127.0.0.1:56690_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=44
[junit4] 2> 652939 DEBUG (zkCallback-4152-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 652939 DEBUG (zkCallback-4152-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:56690_solr, 127.0.0.1:46101_solr, 127.0.0.1:38382_solr]
[junit4] 2> 652939 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 652940 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@2d52a46e{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 652940 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2069808969
[junit4] 2> 652940 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 652940 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6258f746: rootName = solr_38382, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@54c4d6f7
[junit4] 2> 652941 INFO (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 652941 DEBUG (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 3
[junit4] 2> 652947 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 652947 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2e8afa17: rootName = solr_38382, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7456463e
[junit4] 2> 652948 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 652948 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 652951 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 652951 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2ada23af: rootName = solr_38382, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1e05e42f
[junit4] 2> 652952 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:38382_solr
[junit4] 2> 652952 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 652952 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 652952 INFO (zkCallback-4164-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 652952 INFO (zkCallback-4153-thread-1-processing-n:127.0.0.1:46101_solr) [n:127.0.0.1:46101_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 652953 INFO (zkCallback-4172-thread-1-processing-n:127.0.0.1:38382_solr) [n:127.0.0.1:38382_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 652953 INFO (zkCallback-4152-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 652953 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@294bb8db{/solr,null,UNAVAILABLE}
[junit4] 2> 652953 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 653942 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 653942 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:38382_solr
[junit4] 2> 654465 INFO (qtp1419071156-12817) [n:127.0.0.1:46101_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 654467 INFO (qtp1500030909-12810) [n:127.0.0.1:56690_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 654468 DEBUG (qtp1419071156-12819) [n:127.0.0.1:46101_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 654468 DEBUG (zkCallback-4152-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 654469 DEBUG (zkCallback-4152-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:56690_solr, 127.0.0.1:46101_solr]
[junit4] 2> 654469 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 654469 INFO (qtp1419071156-12819) [n:127.0.0.1:46101_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=7
[junit4] 2> 654479 INFO (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 654479 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:38382_solr
[junit4] 2> 654479 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 654479 DEBUG (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 654479 DEBUG (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:38382_solr]
[junit4] 2> 654479 DEBUG (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4225e05c1f979bTbofhf7ny2uz4xbcdebdpqu1yw",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":18618994011379611,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18618994011379611],
[junit4] 2> "nodeNames":["127.0.0.1:38382_solr"]}}
[junit4] 2> 654480 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 654480 DEBUG (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_restore_trigger]
[junit4] 2> 654480 INFO (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 654486 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"4225e05c1f979bTbofhf7ny2uz4xbcdebdpqu1yw",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":18618994011379611,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18618994011379611],
[junit4] 2> "_enqueue_time_":18618994012499732,
[junit4] 2> "nodeNames":["127.0.0.1:38382_solr"]}}
[junit4] 2> 654486 INFO (TEST-TriggerIntegrationTest.testNodeLostTriggerRestoreState-seed#[2136A2C8734F856E]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTriggerRestoreState
[junit4] 2> 654499 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_restore_trigger after 5000ms
[junit4] 2> 654499 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTrigger
[junit4] 2> 654499 DEBUG (zkCallback-4152-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 654499 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 654500 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 5
[junit4] 2> 654501 INFO (qtp1419071156-12818) [n:127.0.0.1:46101_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 654502 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 15 ms for event id=4225e05c1f979bTbofhf7ny2uz4xbcdebdpqu1yw
[junit4] 2> 654505 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 654505 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 654505 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 654508 INFO (qtp1419071156-12818) [n:127.0.0.1:46101_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 654509 INFO (qtp1419071156-12820) [n:127.0.0.1:46101_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 654510 INFO (qtp1500030909-12808) [n:127.0.0.1:56690_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 654510 DEBUG (qtp1419071156-12819) [n:127.0.0.1:46101_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 654510 INFO (qtp1419071156-12819) [n:127.0.0.1:46101_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 654516 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 654516 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:56690_solr, 127.0.0.1:46101_solr]
[junit4] 2> 654516 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 654517 INFO (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 654517 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 654518 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 654518 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 654521 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@2dc06888{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 654521 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=734533269
[junit4] 2> 654521 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 654522 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2a2234a9: rootName = solr_46101, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6d890382
[junit4] 2> 654534 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 654535 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6fdcfa4: rootName = solr_46101, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7456463e
[junit4] 2> 654538 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 654538 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3bc0e8a0: rootName = solr_46101, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1e05e42f
[junit4] 2> 654540 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:46101_solr
[junit4] 2> 654541 INFO (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 654541 INFO (zkCallback-4153-thread-1-processing-n:127.0.0.1:46101_solr) [n:127.0.0.1:46101_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 654542 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 654542 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 654546 INFO (zkCallback-4164-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 654547 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@522992b5{/solr,null,UNAVAILABLE}
[junit4] 2> 654547 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 655520 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
[junit4] 2> 655520 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:46101_solr
[junit4] 2> 656503 INFO (OverseerCollectionConfigSetProcessor-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 656521 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
[junit4] 2> 657521 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
[junit4] 2> 658521 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 1
[junit4] 2> 658522 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:46101_solr]
[junit4] 2> 658522 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4225e09a30f95eTbofhf7ny2uz4xbcdebdpqu1yy",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":18618995052706142,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18618995052706142],
[junit4] 2> "nodeNames":["127.0.0.1:46101_solr"]}}
[junit4] 2> 658522 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger]
[junit4] 2> 658522 INFO (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 658523 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"4225e09a30f95eTbofhf7ny2uz4xbcdebdpqu1yy",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":18618995052706142,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18618995052706142],
[junit4] 2> "_enqueue_time_":18618998054400217,
[junit4] 2> "nodeNames":["127.0.0.1:46101_solr"]}}
[junit4] 2> 658524 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 5000ms
[junit4] 2> 658524 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=4225e09a30f95eTbofhf7ny2uz4xbcdebdpqu1yy
[junit4] 2> 658527 INFO (qtp1500030909-12811) [n:127.0.0.1:56690_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 658528 DEBUG (qtp1500030909-12810) [n:127.0.0.1:56690_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 658528 INFO (qtp1500030909-12810) [n:127.0.0.1:56690_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 658528 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 658529 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:56690_solr]
[junit4] 2> 658529 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 658529 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 658529 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 660529 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[2136A2C8734F856E]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTrigger
[junit4] 2> 660539 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
[junit4] 2> 660539 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 8
[junit4] 2> 660539 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 660540 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 660541 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 660541 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 660541 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 660541 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 660542 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 660542 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 660542 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 660542 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@eb47e52{/solr,null,AVAILABLE}
[junit4] 2> 660543 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@39f81b5c{SSL,[ssl, http/1.1]}{127.0.0.1:38517}
[junit4] 2> 660543 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.Server Started @663090ms
[junit4] 2> 660543 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38517}
[junit4] 2> 660543 ERROR (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 660543 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 660543 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 660543 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 660543 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T16:38:09.412Z
[junit4] 2> 660544 INFO (zkConnectionManagerCallback-4177-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 660545 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 660551 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56871/solr
[junit4] 2> 660552 INFO (zkConnectionManagerCallback-4181-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 660554 INFO (zkConnectionManagerCallback-4183-thread-1-processing-n:127.0.0.1:38517_solr) [n:127.0.0.1:38517_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 660558 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:38517_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 660560 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:38517_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 660561 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:38517_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38517_solr
[junit4] 2> 660562 INFO (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 660562 INFO (zkCallback-4164-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 660564 INFO (zkCallback-4182-thread-1-processing-n:127.0.0.1:38517_solr) [n:127.0.0.1:38517_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 660614 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:38517_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38517.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 660622 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:38517_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38517.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 660622 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:38517_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38517.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 660624 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:38517_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_2136A2C8734F856E-001/tempDir-001/node4/.
[junit4] 2> 660670 INFO (qtp1943903053-12904) [n:127.0.0.1:38517_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 660671 INFO (qtp1500030909-12807) [n:127.0.0.1:56690_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 660671 DEBUG (qtp1500030909-12808) [n:127.0.0.1:56690_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 660672 INFO (qtp1500030909-12808) [n:127.0.0.1:56690_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 660672 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 660675 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:38517_solr, 127.0.0.1:56690_solr]
[junit4] 2> 660675 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 660675 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 660676 INFO (qtp1943903053-12902) [n:127.0.0.1:38517_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 660678 INFO (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 660679 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 660682 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 660682 DEBUG (ScheduledTrigger-2718-thread-4) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 660682 DEBUG (ScheduledTrigger-2718-thread-4) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 660684 INFO (qtp1500030909-12808) [n:127.0.0.1:56690_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=5
[junit4] 2> 660684 DEBUG (qtp1943903053-12905) [n:127.0.0.1:38517_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 660684 INFO (qtp1943903053-12905) [n:127.0.0.1:38517_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=10
[junit4] 2> 660685 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 660685 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:38517_solr, 127.0.0.1:56690_solr]
[junit4] 2> 660685 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 660685 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:38517_solr, 127.0.0.1:56690_solr]
[junit4] 2> 660685 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 660685 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 660686 INFO (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 660688 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 660689 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 660692 DEBUG (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 660692 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 660692 DEBUG (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 660696 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 660696 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 660696 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 660696 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@161a0478{/solr,null,AVAILABLE}
[junit4] 2> 660697 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5ce3b67f{SSL,[ssl, http/1.1]}{127.0.0.1:44938}
[junit4] 2> 660697 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.Server Started @663245ms
[junit4] 2> 660697 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44938}
[junit4] 2> 660697 ERROR (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 660697 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 660697 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 660697 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 660697 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T16:38:09.566Z
[junit4] 2> 660704 INFO (zkConnectionManagerCallback-4187-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 660704 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 660712 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56871/solr
[junit4] 2> 660713 INFO (zkConnectionManagerCallback-4191-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 660714 INFO (zkConnectionManagerCallback-4193-thread-1-processing-n:127.0.0.1:44938_solr) [n:127.0.0.1:44938_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 660720 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:44938_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 660721 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:44938_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 660723 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:44938_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44938_solr
[junit4] 2> 660732 INFO (zkCallback-4182-thread-1-processing-n:127.0.0.1:38517_solr) [n:127.0.0.1:38517_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 660732 INFO (zkCallback-4164-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 660732 INFO (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 660735 INFO (zkCallback-4192-thread-1-processing-n:127.0.0.1:44938_solr) [n:127.0.0.1:44938_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 660793 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:44938_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44938.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 660802 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:44938_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44938.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 660802 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:44938_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44938.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@559383cb
[junit4] 2> 660803 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [n:127.0.0.1:44938_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_2136A2C8734F856E-001/tempDir-001/node5/.
[junit4] 2> 661684 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 661684 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 661684 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:44938_solr at time 18619001216685402
[junit4] 2> 661684 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:44938_solr] added at times [18619001216685402], now=18619001216847152
[junit4] 2> 661685 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4225e20997d55aTbofhf7ny2uz4xbcdebdpqu1z0",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":18619001216685402,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18619001216685402],
[junit4] 2> "nodeNames":["127.0.0.1:44938_solr"]}}
[junit4] 2> 661685 DEBUG (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger2, node_added_trigger1]
[junit4] 2> 661685 INFO (ScheduledTrigger-2718-thread-2) [n:127.0.0.1:56690_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 661685 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"4225e20997d55aTbofhf7ny2uz4xbcdebdpqu1z0",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":18619001216685402,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18619001216685402],
[junit4] 2> "_enqueue_time_":18619001217244184,
[junit4] 2> "nodeNames":["127.0.0.1:44938_solr"]}}
[junit4] 2> 661686 INFO (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_added_trigger1
[junit4] 2> 661686 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 661686 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 661687 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=4225e20997d55aTbofhf7ny2uz4xbcdebdpqu1z0
[junit4] 2> 666687 DEBUG (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 666687 DEBUG (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 666687 DEBUG (ScheduledTrigger-2718-thread-3) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 666687 DEBUG (ScheduledTrigger-2718-thread-3) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 666688 DEBUG (ScheduledTrigger-2718-thread-3) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:44938_solr at time 18619006220034395
[junit4] 2> 666688 DEBUG (ScheduledTrigger-2718-thread-3) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:44938_solr] added at times [18619006220034395], now=18619006220075685
[junit4] 2> 666688 DEBUG (ScheduledTrigger-2718-thread-3) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4225e333d0e15bTbofhf7ny2uz4xbcdebdpqu1z2",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":18619006220034395,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18619006220034395],
[junit4] 2> "nodeNames":["127.0.0.1:44938_solr"]}}
[junit4] 2> 666688 DEBUG (ScheduledTrigger-2718-thread-3) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger2, node_added_trigger1]
[junit4] 2> 666688 INFO (ScheduledTrigger-2718-thread-3) [n:127.0.0.1:56690_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 666688 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"4225e333d0e15bTbofhf7ny2uz4xbcdebdpqu1z2",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":18619006220034395,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18619006220034395],
[junit4] 2> "_enqueue_time_":18619006220280012,
[junit4] 2> "nodeNames":["127.0.0.1:44938_solr"]}}
[junit4] 2> 666689 INFO (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.TriggerIntegrationTest last action at 1518367090555000000 current time = 1518367095557000000
[junit4] 2> real diff: 5002000000
[junit4] 2> min diff: 4998000000
[junit4] 2> 666689 INFO (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_added_trigger2
[junit4] 2> 666689 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 666689 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 666689 DEBUG (AutoscalingActionExecutor-2719-thread-1-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=4225e333d0e15bTbofhf7ny2uz4xbcdebdpqu1z2
[junit4] 2> 666693 INFO (qtp1943903053-12906) [n:127.0.0.1:38517_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 666714 INFO (qtp2067546717-12930) [n:127.0.0.1:44938_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 666718 INFO (qtp1500030909-12811) [n:127.0.0.1:56690_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 666718 DEBUG (qtp1943903053-12904) [n:127.0.0.1:38517_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 666718 INFO (qtp1943903053-12904) [n:127.0.0.1:38517_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=28
[junit4] 2> 666719 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 666719 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:38517_solr, 127.0.0.1:56690_solr, 127.0.0.1:44938_solr]
[junit4] 2> 666719 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 666719 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:38517_solr, 127.0.0.1:56690_solr, 127.0.0.1:44938_solr]
[junit4] 2> 666719 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:38517_solr, 127.0.0.1:56690_solr, 127.0.0.1:44938_solr]
[junit4] 2> 666719 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 666719 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 666720 INFO (qtp1943903053-12902) [n:127.0.0.1:38517_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 666721 INFO (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 666721 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 666722 DEBUG (ScheduledTrigger-2718-thread-1) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 3
[junit4] 2> 666722 INFO (qtp2067546717-12927) [n:127.0.0.1:44938_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 666723 INFO (qtp1500030909-12808) [n:127.0.0.1:56690_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 666723 DEBUG (qtp1943903053-12905) [n:127.0.0.1:38517_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 666724 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 666724 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:38517_solr, 127.0.0.1:56690_solr, 127.0.0.1:44938_solr]
[junit4] 2> 666724 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 666724 INFO (qtp1943903053-12905) [n:127.0.0.1:38517_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 666724 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:38517_solr, 127.0.0.1:56690_solr, 127.0.0.1:44938_solr]
[junit4] 2> 666725 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:38517_solr, 127.0.0.1:56690_solr, 127.0.0.1:44938_solr]
[junit4] 2> 666725 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 666725 DEBUG (zkCallback-4152-thread-2-processing-n:127.0.0.1:56690_solr) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:38517_solr, 127.0.0.1:56690_solr, 127.0.0.1:44938_solr]
[junit4] 2> 666725 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 666726 INFO (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 666727 DEBUG (OverseerAutoScalingTriggerThread-73277808235708422-127.0.0.1:56690_solr-n_0000000000) [n:127.0.0.1:56690_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 666727 DEBUG (ScheduledTrigger-2718-thread-4) [n:127.0.0.1:56690_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 3
[junit4] 2> 666727 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5ce3b67f{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 666727 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1553389149
[junit4] 2> 666727 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 666728 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4cf00804: rootName = solr_44938, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@13e1428c
[junit4] 2> 666734 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 666734 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@75801dcd: rootName = solr_44938, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7456463e
[junit4] 2> 666738 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 666738 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[2136A2C8734F856E])
[...truncated too long message...]
Writer is ready to be used.
[junit4] 2> 1048507 INFO (indexFetcher-4253-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ffa6f55[collection1] main]
[junit4] 2> 1048509 INFO (searcherExecutor-4249-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1ffa6f55[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):c493) Uninverting(_1(8.0.0):c1)))}
[junit4] 2> 1048532 INFO (qtp386817671-10868) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=494 status=0 QTime=0
[junit4] 2> 1048533 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.h.TestReplicationHandler Waited for 900ms and found 494 docs
[junit4] 2> 1048533 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.SolrTestCaseJ4 ###Ending doTestStopPoll
[junit4] 2> 1048533 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1b2f5481{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1048534 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1954106589
[junit4] 2> 1048534 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1048534 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@58bc02a8: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@415ecf4b
[junit4] 2> 1048539 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1048539 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f5ece2e: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@29e515c4
[junit4] 2> 1048542 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1048542 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@589635f: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@16703010
[junit4] 2> 1048543 INFO (coreCloseExecutor-4256-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7abaabc2
[junit4] 2> 1048543 INFO (coreCloseExecutor-4256-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=2059054018
[junit4] 2> 1048543 INFO (coreCloseExecutor-4256-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4d046ce5: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@22b6e8be
[junit4] 2> 1048553 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@30194921{/solr,null,UNAVAILABLE}
[junit4] 2> 1048553 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1048554 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@2486c5f3{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1048554 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1800635427
[junit4] 2> 1048554 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1048554 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4fe3b7ef: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5f58d014
[junit4] 2> 1048561 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1048561 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6343192c: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@29e515c4
[junit4] 2> 1048568 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1048568 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5e50cd73: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@16703010
[junit4] 2> 1048573 INFO (coreCloseExecutor-4258-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1619414
[junit4] 2> 1048573 INFO (coreCloseExecutor-4258-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=23172116
[junit4] 2> 1048573 INFO (coreCloseExecutor-4258-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@14235abd: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@752afc72
[junit4] 2> 1048581 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5956d1a7{/solr,null,UNAVAILABLE}
[junit4] 2> 1048581 INFO (TEST-TestReplicationHandler.doTestStopPoll-seed#[2136A2C8734F856E]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.handler.TestReplicationHandler_2136A2C8734F856E-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {name=PostingsFormat(name=Direct), id=PostingsFormat(name=Direct), newname=PostingsFormat(name=Direct)}, docValues:{}, maxPointsInLeafNode=1689, maxMBSortInHeap=7.53234471548032, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@1282af94), locale=en-GB, timezone=Asia/Hebron
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=233381640,total=531103744
[junit4] 2> NOTE: All tests run in this JVM: [TestSimpleTrackingShardHandler, BasicZkTest, TestQueryWrapperFilter, InfoHandlerTest, TimeRoutedAliasUpdateProcessorTest, SampleTest, TestConfigOverlay, OverseerTest, TestPushWriter, BooleanFieldTest, TestIntervalFaceting, TestLeaderElectionZkExpiry, HdfsRecoveryZkTest, TestLRUCache, VersionInfoTest, CollectionStateFormat2Test, SolrInfoBeanTest, TestMultiValuedNumericRangeQuery, TestDynamicLoading, RuleEngineTest, TestStressRecovery, TestPullReplica, CreateRoutedAliasTest, DistributedMLTComponentTest, DocValuesNotIndexedTest, SolrSlf4jReporterTest, DistribJoinFromCollectionTest, RequiredFieldsTest, LeaderInitiatedRecoveryOnShardRestartTest, TestInPlaceUpdatesDistrib, TestFastLRUCache, IndexSchemaTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestBinaryResponseWriter, DeleteStatusTest, TestAnalyzeInfixSuggestions, DirectUpdateHandlerOptimizeTest, TestXIncludeConfig, TestSolrJ, TestHdfsBackupRestoreCore, ScriptEngineTest, MoveReplicaTest, TestRandomRequestDistribution, TestRecovery, RequestLoggingTest, BlockJoinFacetDistribTest, ReplicaListTransformerTest, TestTolerantUpdateProcessorCloud, SyncSliceTest, TestNumericTokenStream, TestUseDocValuesAsStored2, DistributedFacetPivotWhiteBoxTest, TestCharFilters, TestSerializedLuceneMatchVersion, QueryElevationComponentTest, TestManagedSynonymFilterFactory, ZkStateReaderTest, TermsComponentTest, TestZkChroot, JsonLoaderTest, TestFieldCacheReopen, HighlighterTest, PeerSyncTest, TestStressVersions, DeleteNodeTest, TestGenericDistributedQueue, DocumentBuilderTest, BasicFunctionalityTest, TestPolicyCloud, AsyncCallRequestStatusResponseTest, AddBlockUpdateTest, TestShardHandlerFactory, GraphQueryTest, RemoteQueryErrorTest, LeaderElectionContextKeyTest, TestFilteredDocIdSet, NodeLostTriggerTest, DistributedSuggestComponentTest, TestCollectionAPI, TestScoreJoinQPNoScore, SystemInfoHandlerTest, OverseerModifyCollectionTest, TestComputePlanAction, TestReplicationHandler]
[junit4] Completed [287/776 (2!)] on J1 in 96.12s, 16 tests, 1 error, 1 skipped <<< FAILURES!
[...truncated 49925 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2311 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2311/
4 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd: 1) Thread[id=120, name=qtp1706926551-120, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd:
1) Thread[id=120, name=qtp1706926551-120, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([A746474AAE52596C]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=120, name=qtp1706926551-120, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=120, name=qtp1706926551-120, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([A746474AAE52596C]:0)
FAILED: org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost
Error Message:
The operations computed by ComputePlanAction should not be null SolrClientNodeStateProvider.DEBUG{AFTER_ACTION=[compute_plan, null], BEFORE_ACTION=[compute_plan, null]}
Stack Trace:
java.lang.AssertionError: The operations computed by ComputePlanAction should not be null SolrClientNodeStateProvider.DEBUG{AFTER_ACTION=[compute_plan, null], BEFORE_ACTION=[compute_plan, null]}
at __randomizedtesting.SeedInfo.seed([81786C5B037118A0:B1B88DD98B03F9FC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost(ComputePlanActionTest.java:291)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testSetProperties
Error Message:
Expected 8 triggers but found: [x0, x1, x2, x3, x4, x5, x6] expected:<8> but was:<7>
Stack Trace:
java.lang.AssertionError: Expected 8 triggers but found: [x0, x1, x2, x3, x4, x5, x6] expected:<8> but was:<7>
at __randomizedtesting.SeedInfo.seed([81786C5B037118A0:EA1CBB12B05C8D24]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testSetProperties(TriggerIntegrationTest.java:1322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12505 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.ComputePlanActionTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_81786C5B037118A0-001/init-core-data-001
[junit4] 2> 666957 WARN (SUITE-ComputePlanActionTest-seed#[81786C5B037118A0]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 666957 INFO (SUITE-ComputePlanActionTest-seed#[81786C5B037118A0]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 666958 INFO (SUITE-ComputePlanActionTest-seed#[81786C5B037118A0]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 666958 INFO (SUITE-ComputePlanActionTest-seed#[81786C5B037118A0]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 666959 INFO (SUITE-ComputePlanActionTest-seed#[81786C5B037118A0]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_81786C5B037118A0-001/tempDir-001
[junit4] 2> 666959 INFO (SUITE-ComputePlanActionTest-seed#[81786C5B037118A0]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 666959 INFO (Thread-2135) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 666959 INFO (Thread-2135) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 666965 ERROR (Thread-2135) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 667059 INFO (SUITE-ComputePlanActionTest-seed#[81786C5B037118A0]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:37378
[junit4] 2> 667061 INFO (zkConnectionManagerCallback-1561-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 667064 INFO (jetty-launcher-1558-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 667069 INFO (jetty-launcher-1558-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 667069 INFO (jetty-launcher-1558-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 667069 INFO (jetty-launcher-1558-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 667069 INFO (jetty-launcher-1558-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@43182e94{/solr,null,AVAILABLE}
[junit4] 2> 667069 INFO (jetty-launcher-1558-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@54bbbe13{HTTP/1.1,[http/1.1]}{127.0.0.1:42838}
[junit4] 2> 667069 INFO (jetty-launcher-1558-thread-1) [ ] o.e.j.s.Server Started @668826ms
[junit4] 2> 667069 INFO (jetty-launcher-1558-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42838}
[junit4] 2> 667070 ERROR (jetty-launcher-1558-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 667070 INFO (jetty-launcher-1558-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 667070 INFO (jetty-launcher-1558-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 667070 INFO (jetty-launcher-1558-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 667070 INFO (jetty-launcher-1558-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T13:29:58.622Z
[junit4] 2> 667078 INFO (zkConnectionManagerCallback-1563-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 667078 INFO (jetty-launcher-1558-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 667085 INFO (jetty-launcher-1558-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37378/solr
[junit4] 2> 667088 INFO (zkConnectionManagerCallback-1567-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 667093 INFO (zkConnectionManagerCallback-1569-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 667137 INFO (jetty-launcher-1558-thread-1) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 667138 INFO (jetty-launcher-1558-thread-1) [n:127.0.0.1:42838_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42838_solr
[junit4] 2> 667138 INFO (jetty-launcher-1558-thread-1) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer Overseer (id=73277068866420739-127.0.0.1:42838_solr-n_0000000000) starting
[junit4] 2> 667143 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 667149 INFO (jetty-launcher-1558-thread-1) [n:127.0.0.1:42838_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42838_solr
[junit4] 2> 667157 INFO (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 667157 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 667158 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 667159 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:42838_solr]
[junit4] 2> 667159 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 667159 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 667160 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 667160 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 667161 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 667225 INFO (jetty-launcher-1558-thread-1) [n:127.0.0.1:42838_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42838.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac5a0ba
[junit4] 2> 667232 INFO (jetty-launcher-1558-thread-1) [n:127.0.0.1:42838_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42838.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac5a0ba
[junit4] 2> 667232 INFO (jetty-launcher-1558-thread-1) [n:127.0.0.1:42838_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42838.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac5a0ba
[junit4] 2> 667233 INFO (jetty-launcher-1558-thread-1) [n:127.0.0.1:42838_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_81786C5B037118A0-001/tempDir-001/node1/.
[junit4] 2> 667255 INFO (zkConnectionManagerCallback-1573-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 667258 INFO (zkConnectionManagerCallback-1577-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 667259 INFO (SUITE-ComputePlanActionTest-seed#[81786C5B037118A0]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 667260 INFO (SUITE-ComputePlanActionTest-seed#[81786C5B037118A0]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37378/solr ready
[junit4] 2> 667276 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAdded
[junit4] 2> 667277 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 667278 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 667279 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 667280 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 667280 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 667281 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 667286 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 667289 DEBUG (qtp1528890485-7222) [n:127.0.0.1:42838_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 667289 INFO (qtp1528890485-7222) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 667290 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 667290 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 667290 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 667291 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 667292 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 667293 INFO (qtp1528890485-7219) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 667293 DEBUG (qtp1528890485-7220) [n:127.0.0.1:42838_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 667294 INFO (qtp1528890485-7220) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 667295 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 667295 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 667295 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 667295 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 667296 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 667296 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 667296 DEBUG (qtp1528890485-7221) [n:127.0.0.1:42838_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 667297 INFO (qtp1528890485-7221) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 667297 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 667300 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:42838_solr]
[junit4] 2> 667300 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.ComputePlanActionTest$AssertingTriggerAction}], enabled=true}
[junit4] 2> 667300 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 667301 INFO (qtp1528890485-7219) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 667301 DEBUG (qtp1528890485-7220) [n:127.0.0.1:42838_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 667301 INFO (qtp1528890485-7220) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 667301 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 667302 INFO (qtp1528890485-7221) [n:127.0.0.1:42838_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=testNodeAdded&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 667302 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:42838_solr]
[junit4] 2> 667302 DEBUG (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.ComputePlanActionTest$AssertingTriggerAction}], enabled=true}
[junit4] 2> 667303 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 667303 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 5
[junit4] 2> 667303 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 667303 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 667303 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 667304 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 667304 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 667305 INFO (OverseerThreadFactory-2609-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testNodeAdded
[junit4] 2> 667306 WARN (OverseerThreadFactory-2609-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 2 on collection testNodeAdded is higher than the number of Solr instances currently live or live and part of your createNodeSet(1). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 667307 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 667309 INFO (OverseerThreadFactory-2609-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18607709524 sessionWrapper.createTime 18607709522519966, this.sessionWrapper.createTime 18607709522519966
[junit4] 2> 667312 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"testNodeAdded",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "collection.configName":"conf",
[junit4] 2> "numShards":"1",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 667312 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ClusterStateMutator building a new cName: testNodeAdded
[junit4] 2> 667313 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/testNodeAdded/state.json
[junit4] 2> 667415 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 667415 INFO (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 667416 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 667416 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 667418 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 667418 INFO (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 667419 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 667419 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 667519 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 0
[junit4] 2> 667620 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n2&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 667621 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n1&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 667621 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 667626 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 667626 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 667626 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"testNodeAdded_shard1_replica_n1","base_url":"http://127.0.0.1:42838/solr","node_name":"127.0.0.1:42838_solr","state":"down","type":"NRT"}
[junit4] 2> 667626 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 667627 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 667627 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 667627 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node4:{"core":"testNodeAdded_shard1_replica_n2","base_url":"http://127.0.0.1:42838/solr","node_name":"127.0.0.1:42838_solr","state":"down","type":"NRT"}
[junit4] 2> 667627 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 667727 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 1
[junit4] 2> 667728 INFO (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
[junit4] 2> 667728 INFO (zkCallback-1568-thread-2-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
[junit4] 2> 668306 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 668307 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 668635 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 668636 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 668652 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.IndexSchema [testNodeAdded_shard1_replica_n2] Schema name=minimal
[junit4] 2> 668653 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.IndexSchema [testNodeAdded_shard1_replica_n1] Schema name=minimal
[junit4] 2> 668654 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 668655 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'testNodeAdded_shard1_replica_n2' using configuration from collection testNodeAdded, trusted=true
[junit4] 2> 668657 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42838.solr.core.testNodeAdded.shard1.replica_n2' (registry 'solr.core.testNodeAdded.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac5a0ba
[junit4] 2> 668657 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 668657 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrCore [[testNodeAdded_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_81786C5B037118A0-001/tempDir-001/node1/testNodeAdded_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_81786C5B037118A0-001/tempDir-001/node1/./testNodeAdded_shard1_replica_n2/data/]
[junit4] 2> 668657 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 668657 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testNodeAdded_shard1_replica_n1' using configuration from collection testNodeAdded, trusted=true
[junit4] 2> 668661 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42838.solr.core.testNodeAdded.shard1.replica_n1' (registry 'solr.core.testNodeAdded.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac5a0ba
[junit4] 2> 668661 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 668661 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrCore [[testNodeAdded_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_81786C5B037118A0-001/tempDir-001/node1/testNodeAdded_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_81786C5B037118A0-001/tempDir-001/node1/./testNodeAdded_shard1_replica_n1/data/]
[junit4] 2> 668701 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 668701 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 668701 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 668701 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 668702 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 668702 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 668703 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 668703 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 668704 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3e1df0f5[testNodeAdded_shard1_replica_n1] main]
[junit4] 2> 668705 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 668705 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@5322c24f[testNodeAdded_shard1_replica_n2] main]
[junit4] 2> 668705 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 668706 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 668707 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 668707 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 668707 INFO (searcherExecutor-2613-thread-1-processing-n:127.0.0.1:42838_solr x:testNodeAdded_shard1_replica_n1 s:shard1 c:testNodeAdded r:core_node3) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.SolrCore [testNodeAdded_shard1_replica_n1] Registered new searcher Searcher@3e1df0f5[testNodeAdded_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 668707 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592111451612381184
[junit4] 2> 668707 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 668708 INFO (searcherExecutor-2612-thread-1-processing-n:127.0.0.1:42838_solr x:testNodeAdded_shard1_replica_n2 s:shard1 c:testNodeAdded r:core_node4) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SolrCore [testNodeAdded_shard1_replica_n2] Registered new searcher Searcher@5322c24f[testNodeAdded_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 668708 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592111451613429760
[junit4] 2> 668715 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.ZkShardTerms Failed to save terms, version is not match, retrying
[junit4] 2> 668719 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 668719 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 668719 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42838/solr/testNodeAdded_shard1_replica_n2/
[junit4] 2> 668719 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded"} current state version: 0
[junit4] 2> 668720 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testNodeAdded_shard1_replica_n2 url=http://127.0.0.1:42838/solr START replicas=[http://127.0.0.1:42838/solr/testNodeAdded_shard1_replica_n1/] nUpdates=100
[junit4] 2> 668722 INFO (qtp1528890485-7226) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.c.S.Request [testNodeAdded_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 668820 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 2
[junit4] 2> 668820 INFO (zkCallback-1568-thread-3-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
[junit4] 2> 668820 INFO (zkCallback-1568-thread-2-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
[junit4] 2> 669020 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=testNodeAdded_shard1_replica_n2 url=http://127.0.0.1:42838/solr DONE. We have no versions. sync failed.
[junit4] 2> 669020 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 669020 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 669020 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 669023 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42838/solr/testNodeAdded_shard1_replica_n2/ shard1
[junit4] 2> 669024 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "state":"active"} current state version: 0
[junit4] 2> 669124 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 3
[junit4] 2> 669125 INFO (zkCallback-1568-thread-3-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
[junit4] 2> 669125 INFO (zkCallback-1568-thread-2-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
[junit4] 2> 669174 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 669175 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 669175 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 669175 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node4:{"core":"testNodeAdded_shard1_replica_n2","base_url":"http://127.0.0.1:42838/solr","node_name":"127.0.0.1:42838_solr","state":"active","type":"NRT","leader":"true"}
[junit4] 2> 669175 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/4)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true"}}}}}
[junit4] 2> 669176 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node4 x:testNodeAdded_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n2&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1555
[junit4] 2> 669276 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 4
[junit4] 2> 669276 INFO (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
[junit4] 2> 669276 INFO (zkCallback-1568-thread-2-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
[junit4] 2> 669307 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 669307 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 669720 INFO (qtp1528890485-7223) [n:127.0.0.1:42838_solr c:testNodeAdded s:shard1 r:core_node3 x:testNodeAdded_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=testNodeAdded_shard1_replica_n1&action=CREATE&numShards=1&collection=testNodeAdded&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2099
[junit4] 2> 669721 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 669721 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"testNodeAdded",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 669721 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"testNodeAdded_shard1_replica_n1","base_url":"http://127.0.0.1:42838/solr","node_name":"127.0.0.1:42838_solr","state":"active","type":"NRT"}
[junit4] 2> 669721 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/5)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"2",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true"}}}}}
[junit4] 2> 669725 INFO (qtp1528890485-7221) [n:127.0.0.1:42838_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 669822 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/testNodeAdded/state.json version: 5
[junit4] 2> 669822 INFO (zkCallback-1568-thread-2-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
[junit4] 2> 669822 INFO (zkCallback-1568-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testNodeAdded/state.json] for collection [testNodeAdded] has occurred - updating... (live nodes size: [1])
[junit4] 2> 670307 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 670307 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 670725 INFO (qtp1528890485-7221) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=testNodeAdded&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3423
[junit4] 2> 670732 INFO (qtp1528890485-7219) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.testNodeAdded.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.testNodeAdded.shard1.replica_n2:INDEX.sizeInBytes} status=0 QTime=1
[junit4] 2> 670735 INFO (qtp1528890485-7224) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=1
[junit4] 2> 670736 DEBUG (qtp1528890485-7220) [n:127.0.0.1:42838_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 670737 INFO (qtp1528890485-7220) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=9
[junit4] 2> 670738 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 670740 DEBUG (zkCallback-1568-thread-2-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 670740 DEBUG (zkCallback-1568-thread-2-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:42838_solr]
[junit4] 2> 670740 DEBUG (zkCallback-1568-thread-2-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=compute_plan, class=solr.ComputePlanAction}, {name=test, class=org.apache.solr.cloud.autoscaling.ComputePlanActionTest$AssertingTriggerAction}], enabled=true}
[junit4] 2> 670740 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 670743 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 670743 DEBUG (OverseerAutoScalingTriggerThread-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 670745 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 670745 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 670745 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 670746 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7bf23c5c{/solr,null,AVAILABLE}
[junit4] 2> 670746 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@628572b1{HTTP/1.1,[http/1.1]}{127.0.0.1:40932}
[junit4] 2> 670746 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.e.j.s.Server Started @672502ms
[junit4] 2> 670746 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40932}
[junit4] 2> 670746 ERROR (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 670747 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 670747 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 670747 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 670747 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T13:30:02.299Z
[junit4] 2> 670749 INFO (zkConnectionManagerCallback-1580-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 670750 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 670759 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37378/solr
[junit4] 2> 670761 INFO (zkConnectionManagerCallback-1584-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 670764 INFO (zkConnectionManagerCallback-1586-thread-1-processing-n:127.0.0.1:40932_solr) [n:127.0.0.1:40932_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 670770 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [n:127.0.0.1:40932_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 670773 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [n:127.0.0.1:40932_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 670775 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [n:127.0.0.1:40932_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40932_solr
[junit4] 2> 670776 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:40932_solr"} current state version: 0
[junit4] 2> 670776 DEBUG (OverseerStateUpdate-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:40932_solr
[junit4] 2> 670779 INFO (zkCallback-1576-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 670780 INFO (zkCallback-1568-thread-2-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 670780 INFO (zkCallback-1576-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 670785 INFO (zkCallback-1585-thread-1-processing-n:127.0.0.1:40932_solr) [n:127.0.0.1:40932_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 670836 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [n:127.0.0.1:40932_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40932.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac5a0ba
[junit4] 2> 670845 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [n:127.0.0.1:40932_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40932.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac5a0ba
[junit4] 2> 670845 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [n:127.0.0.1:40932_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40932.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1ac5a0ba
[junit4] 2> 670846 INFO (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [n:127.0.0.1:40932_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.ComputePlanActionTest_81786C5B037118A0-001/tempDir-001/node2/.
[junit4] 2> 671307 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 671307 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 671307 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:40932_solr at time 18607713522923597
[junit4] 2> 671308 INFO (OverseerCollectionConfigSetProcessor-73277068866420739-127.0.0.1:42838_solr-n_0000000000) [n:127.0.0.1:42838_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 672308 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 672309 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 672309 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:40932_solr] added at times [18607713522923597], now=18607714524193627
[junit4] 2> 672309 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"421b9dea9db84dT4ybehnjv8aqtxk3mlj1wf80d8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":18607713522923597,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18607713522923597],
[junit4] 2> "nodeNames":["127.0.0.1:40932_solr"]}}
[junit4] 2> 672310 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger]
[junit4] 2> 672312 INFO (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:42838_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 672313 DEBUG (AutoscalingActionExecutor-2608-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"421b9dea9db84dT4ybehnjv8aqtxk3mlj1wf80d8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":18607713522923597,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18607713522923597],
[junit4] 2> "_enqueue_time_":18607714525562023,
[junit4] 2> "nodeNames":["127.0.0.1:40932_solr"]}}
[junit4] 2> 672313 DEBUG (AutoscalingActionExecutor-2608-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"421b9dea9db84dT4ybehnjv8aqtxk3mlj1wf80d8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":18607713522923597,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18607713522923597],
[junit4] 2> "_enqueue_time_":18607714525562023,
[junit4] 2> "nodeNames":["127.0.0.1:40932_solr"]}} with context properties: {BEFORE_ACTION=[compute_plan]}
[junit4] 2> 672322 INFO (qtp1311237093-7276) [n:127.0.0.1:40932_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 672322 INFO (qtp1528890485-7219) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.testNodeAdded.shard1.replica_n1:INDEX.sizeInBytes&key=solr.core.testNodeAdded.shard1.replica_n2:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 672324 INFO (qtp1528890485-7220) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 672324 INFO (AutoscalingActionExecutor-2608-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.ComputePlanAction Computed Plan: action=MOVEREPLICA&collection=testNodeAdded&targetNode=127.0.0.1:40932_solr&inPlaceMove=true&replica=core_node3
[junit4] 2> 672325 INFO (AutoscalingActionExecutor-2608-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18607714540 sessionWrapper.createTime 18607714539459793, this.sessionWrapper.createTime 18607714539459793
[junit4] 2> 672325 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.c.a.ComputePlanActionTest -------------_ FINAL STATE --------------
[junit4] 2> 672326 DEBUG (AutoscalingActionExecutor-2608-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 5000ms
[junit4] 2> 672326 DEBUG (AutoscalingActionExecutor-2608-thread-1-processing-n:127.0.0.1:42838_solr) [n:127.0.0.1:42838_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 12 ms for event id=421b9dea9db84dT4ybehnjv8aqtxk3mlj1wf80d8
[junit4] 2> 672326 INFO (qtp1311237093-7277) [n:127.0.0.1:40932_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 672326 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.c.a.ComputePlanActionTest * Node values: 127.0.0.1:40932_solr
[junit4] 2> {
[junit4] 2> "node":"127.0.0.1:40932_solr",
[junit4] 2> "cores":0,
[junit4] 2> "port":"40932",
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "ip_1":"1",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"127",
[junit4] 2> "freedisk":51.07687759399414}
[junit4] 2> 672327 INFO (qtp1528890485-7219) [n:127.0.0.1:42838_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 672328 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.c.a.ComputePlanActionTest * Node values: 127.0.0.1:42838_solr
[junit4] 2> {
[junit4] 2> "node":"127.0.0.1:42838_solr",
[junit4] 2> "cores":2,
[junit4] 2> "port":"42838",
[junit4] 2> "host":"127.0.0.1",
[junit4] 2> "ip_1":"1",
[junit4] 2> "ip_3":"0",
[junit4] 2> "ip_2":"0",
[junit4] 2> "ip_4":"127",
[junit4] 2> "freedisk":51.07687759399414}
[junit4] 2> 672328 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.c.a.ComputePlanActionTest * Live nodes: [127.0.0.1:40932_solr, 127.0.0.1:42838_solr]
[junit4] 2> 672328 DEBUG (TEST-ComputePlanActionTest.testNodeAdded-seed#[81786C5B037118A0]) [ ] o.a.s.c.a.ComputePlanActionTest * Collection testNodeAdded state: DocCollection(testNodeAdded//collections/testNodeAdded/state.json/6)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"2",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node4":{
[junit4] 2> "core":"testNodeAdded_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:42838/solr",
[junit4] 2> "node_name":"127.0.0.1:42838_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true"}}}},
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2>
[...truncated too long message...]
2.2} hits=1 status=0 QTime=0
[junit4] 2> 47031 T46 C14 oasc.SolrCore.execute [collection1] webapp=null path=null params={q=id:2&qt=&start=0&rows=20&version=2.2} hits=1 status=0 QTime=0
[junit4] 2> 47031 T46 oas.SolrTestCaseJ4.deleteCore ###deleteCore
[junit4] 2> 47032 T46 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=924785837
[junit4] 2> 47032 T46 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 47032 T46 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@768af3e8: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / null
[junit4] 2> 47032 T46 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 47032 T46 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4f7a20d2: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / null
[junit4] 2> 47032 T46 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 47033 T46 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@291edf08: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / null
[junit4] 2> 47034 T250 x:x:collection1 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@5ce53627
[junit4] 2> 47034 T250 x:x:collection1 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.core.collection1, tag=1558525479
[junit4] 2> 47034 T250 x:x:collection1 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@11d487c9: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / null
[junit4] 2> 47040 T46 oejs.AbstractConnector.doStop Stopped ServerConnector@c7b9c95{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 47041 T46 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=1659423280
[junit4] 2> 47041 T46 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 47041 T46 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 47041 T46 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 47042 T253 x:x:collection1 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1dd5cd0b
[junit4] 2> 47042 T253 x:x:collection1 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.core.collection1, tag=500550923
[junit4] 2> 47052 T46 oejsh.ContextHandler.doStop Stopped o.e.j.s.ServletContextHandler@73a87fa6{/solr,null,UNAVAILABLE}
[junit4] 2> 47052 T46 oejss.HouseKeeper.stopScavenging Stopped scavenging
[junit4] 2> 47055 T46 oas.SolrTestCaseJ4.tearDown ###Ending testFullImportInnerEntity
[junit4] 2> 47055 T46 oas.SolrTestCaseJ4.clearObjectTrackerAndCheckEmpty ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 11, 2018 2:15:21 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 11, 2018 2:15:41 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd:
[junit4] 2> 1) Thread[id=120, name=qtp1706926551-120, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 11, 2018 2:15:41 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=120, name=qtp1706926551-120, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
[junit4] 2> Feb 11, 2018 2:15:44 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=120, name=qtp1706926551-120, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {dbdesc_s=Lucene50(blocksize=128), dbid_s=PostingsFormat(name=MockRandom), id=PostingsFormat(name=MockRandom), desc=Lucene50(blocksize=128)}, docValues:{}, maxPointsInLeafNode=1449, maxMBSortInHeap=6.881502100490428, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2296cb72), locale=ar-LY, timezone=Pacific/Fiji
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=3,free=120060944,total=292028416
[junit4] 2> NOTE: All tests run in this JVM: [TestDateFormatTransformer, TestEphemeralCache, TestDocBuilder, TestDataConfig, TestVariableResolver, TestJdbcDataSource, TestFieldReader, TestURLDataSource, TestRegexTransformer, TestSolrEntityProcessorEndToEnd]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestSolrEntityProcessorEndToEnd -Dtests.seed=A746474AAE52596C -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ar-LY -Dtests.timezone=Pacific/Fiji -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J2 | TestSolrEntityProcessorEndToEnd (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.handler.dataimport.TestSolrEntityProcessorEndToEnd:
[junit4] > 1) Thread[id=120, name=qtp1706926551-120, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([A746474AAE52596C]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=120, name=qtp1706926551-120, state=TIMED_WAITING, group=TGRP-TestSolrEntityProcessorEndToEnd]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([A746474AAE52596C]:0)
[junit4] Completed [38/38 (1!)] on J2 in 61.27s, 7 tests, 2 errors <<< FAILURES!
[...truncated 42599 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2310 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2310/
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testSetProperties
Error Message:
Expected 8 triggers but found: [x0, x1, x2, x3, x4, x5, x6] expected:<8> but was:<7>
Stack Trace:
java.lang.AssertionError: Expected 8 triggers but found: [x0, x1, x2, x3, x4, x5, x6] expected:<8> but was:<7>
at __randomizedtesting.SeedInfo.seed([41AE5BAD79BAFD5:6F7E32F364B63A51]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testSetProperties(TriggerIntegrationTest.java:1322)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13309 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> 1480135 INFO (SUITE-TriggerIntegrationTest-seed#[41AE5BAD79BAFD5]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_41AE5BAD79BAFD5-001/init-core-data-001
[junit4] 2> 1480136 WARN (SUITE-TriggerIntegrationTest-seed#[41AE5BAD79BAFD5]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=5 numCloses=5
[junit4] 2> 1480140 INFO (SUITE-TriggerIntegrationTest-seed#[41AE5BAD79BAFD5]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1480141 INFO (SUITE-TriggerIntegrationTest-seed#[41AE5BAD79BAFD5]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1480142 INFO (SUITE-TriggerIntegrationTest-seed#[41AE5BAD79BAFD5]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_41AE5BAD79BAFD5-001/tempDir-001
[junit4] 2> 1480142 INFO (SUITE-TriggerIntegrationTest-seed#[41AE5BAD79BAFD5]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1480142 INFO (Thread-2517) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1480142 INFO (Thread-2517) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1480143 ERROR (Thread-2517) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1480242 INFO (SUITE-TriggerIntegrationTest-seed#[41AE5BAD79BAFD5]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46415
[junit4] 2> 1480270 INFO (zkConnectionManagerCallback-4981-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1480276 INFO (jetty-launcher-4978-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1480278 INFO (jetty-launcher-4978-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1480278 INFO (jetty-launcher-4978-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1480278 INFO (jetty-launcher-4978-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1480278 INFO (jetty-launcher-4978-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1480279 INFO (jetty-launcher-4978-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@38b1a455{/solr,null,AVAILABLE}
[junit4] 2> 1480280 INFO (jetty-launcher-4978-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@76fd2bcd{SSL,[ssl, http/1.1]}{127.0.0.1:33114}
[junit4] 2> 1480280 INFO (jetty-launcher-4978-thread-1) [ ] o.e.j.s.Server Started @1482070ms
[junit4] 2> 1480280 INFO (jetty-launcher-4978-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33114}
[junit4] 2> 1480281 ERROR (jetty-launcher-4978-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1480281 INFO (jetty-launcher-4978-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1480281 INFO (jetty-launcher-4978-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1480281 INFO (jetty-launcher-4978-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1480281 INFO (jetty-launcher-4978-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T10:39:01.460Z
[junit4] 2> 1480328 INFO (zkConnectionManagerCallback-4983-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1480328 INFO (jetty-launcher-4978-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1480335 INFO (jetty-launcher-4978-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1480335 INFO (jetty-launcher-4978-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1480335 INFO (jetty-launcher-4978-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1480335 INFO (jetty-launcher-4978-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6f83873a{/solr,null,AVAILABLE}
[junit4] 2> 1480336 INFO (jetty-launcher-4978-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7d4c12ac{SSL,[ssl, http/1.1]}{127.0.0.1:46785}
[junit4] 2> 1480336 INFO (jetty-launcher-4978-thread-2) [ ] o.e.j.s.Server Started @1482126ms
[junit4] 2> 1480336 INFO (jetty-launcher-4978-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46785}
[junit4] 2> 1480336 ERROR (jetty-launcher-4978-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1480336 INFO (jetty-launcher-4978-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1480336 INFO (jetty-launcher-4978-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1480336 INFO (jetty-launcher-4978-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1480336 INFO (jetty-launcher-4978-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T10:39:01.515Z
[junit4] 2> 1480341 INFO (jetty-launcher-4978-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46415/solr
[junit4] 2> 1480343 INFO (zkConnectionManagerCallback-4985-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1480344 INFO (jetty-launcher-4978-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1480350 INFO (jetty-launcher-4978-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46415/solr
[junit4] 2> 1480372 INFO (zkConnectionManagerCallback-4989-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1480380 INFO (zkConnectionManagerCallback-4993-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1480381 INFO (zkConnectionManagerCallback-4995-thread-1-processing-n:127.0.0.1:33114_solr) [n:127.0.0.1:33114_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1480382 INFO (zkConnectionManagerCallback-4997-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1480508 INFO (jetty-launcher-4978-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1480509 INFO (jetty-launcher-4978-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46785_solr
[junit4] 2> 1480514 INFO (jetty-launcher-4978-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.Overseer Overseer (id=73276396651216902-127.0.0.1:46785_solr-n_0000000000) starting
[junit4] 2> 1480516 INFO (jetty-launcher-4978-thread-1) [n:127.0.0.1:33114_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1480517 INFO (jetty-launcher-4978-thread-1) [n:127.0.0.1:33114_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33114_solr
[junit4] 2> 1480527 INFO (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1480535 INFO (zkCallback-4994-thread-1-processing-n:127.0.0.1:33114_solr) [n:127.0.0.1:33114_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1480594 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1480595 INFO (jetty-launcher-4978-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46785_solr
[junit4] 2> 1480595 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1480602 INFO (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1480602 INFO (zkCallback-4994-thread-1-processing-n:127.0.0.1:33114_solr) [n:127.0.0.1:33114_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1480610 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:33114_solr, 127.0.0.1:46785_solr]
[junit4] 2> 1480610 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1480610 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1480620 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1480620 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1480621 DEBUG (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1480721 INFO (jetty-launcher-4978-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46785.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1480734 INFO (jetty-launcher-4978-thread-1) [n:127.0.0.1:33114_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33114.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1480741 INFO (jetty-launcher-4978-thread-1) [n:127.0.0.1:33114_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33114.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1480763 INFO (jetty-launcher-4978-thread-1) [n:127.0.0.1:33114_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33114.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1480764 INFO (jetty-launcher-4978-thread-1) [n:127.0.0.1:33114_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_41AE5BAD79BAFD5-001/tempDir-001/node1/.
[junit4] 2> 1480751 INFO (jetty-launcher-4978-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46785.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1480764 INFO (jetty-launcher-4978-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46785.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1480827 INFO (jetty-launcher-4978-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_41AE5BAD79BAFD5-001/tempDir-001/node2/.
[junit4] 2> 1480942 INFO (zkConnectionManagerCallback-5003-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1480974 INFO (zkConnectionManagerCallback-5007-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1480974 INFO (SUITE-TriggerIntegrationTest-seed#[41AE5BAD79BAFD5]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1480975 INFO (SUITE-TriggerIntegrationTest-seed#[41AE5BAD79BAFD5]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46415/solr ready
[junit4] 2> 1481007 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testListeners
[junit4] 2> 1481008 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1481008 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1481008 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 1481038 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1481038 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1481038 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1481071 INFO (qtp916929797-17707) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1481111 INFO (qtp1614980409-17713) [n:127.0.0.1:46785_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1481111 DEBUG (qtp916929797-17702) [n:127.0.0.1:33114_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1481112 INFO (qtp916929797-17702) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=54
[junit4] 2> 1481112 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1481113 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:33114_solr, 127.0.0.1:46785_solr]
[junit4] 2> 1481113 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 1481113 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1481114 INFO (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1481118 DEBUG (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1481118 DEBUG (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1481130 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1481130 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1481149 INFO (qtp916929797-17708) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1481155 INFO (qtp1614980409-17716) [n:127.0.0.1:46785_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1481156 DEBUG (qtp916929797-17704) [n:127.0.0.1:33114_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1481157 INFO (qtp916929797-17704) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=42
[junit4] 2> 1481158 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1481158 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:33114_solr, 127.0.0.1:46785_solr]
[junit4] 2> 1481158 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 1481158 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1481159 INFO (qtp916929797-17706) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1481160 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1481160 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1481162 INFO (qtp1614980409-17718) [n:127.0.0.1:46785_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1481162 DEBUG (qtp916929797-17707) [n:127.0.0.1:33114_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1481163 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1481163 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:33114_solr, 127.0.0.1:46785_solr]
[junit4] 2> 1481163 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 1481163 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1481163 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1481164 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1481164 INFO (qtp916929797-17707) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 1481165 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1481214 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1481214 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1481214 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1481226 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@455b5674{/solr,null,AVAILABLE}
[junit4] 2> 1481227 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@240d24d9{SSL,[ssl, http/1.1]}{127.0.0.1:44593}
[junit4] 2> 1481227 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.Server Started @1483017ms
[junit4] 2> 1481227 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44593}
[junit4] 2> 1481227 ERROR (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1481227 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1481227 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1481227 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1481227 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T10:39:02.406Z
[junit4] 2> 1481245 INFO (zkConnectionManagerCallback-5009-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1481246 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1481276 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46415/solr
[junit4] 2> 1481312 INFO (zkConnectionManagerCallback-5013-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1481360 INFO (zkConnectionManagerCallback-5015-thread-1-processing-n:127.0.0.1:44593_solr) [n:127.0.0.1:44593_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1481364 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:44593_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1481374 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:44593_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1481376 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:44593_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44593_solr
[junit4] 2> 1481377 INFO (zkCallback-4994-thread-1-processing-n:127.0.0.1:33114_solr) [n:127.0.0.1:33114_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1481377 INFO (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1481410 INFO (zkCallback-5006-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1481416 INFO (zkCallback-5014-thread-1-processing-n:127.0.0.1:44593_solr) [n:127.0.0.1:44593_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1481499 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:44593_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44593.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1481509 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:44593_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44593.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1481509 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:44593_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44593.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1481531 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:44593_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_41AE5BAD79BAFD5-001/tempDir-001/node3/.
[junit4] 2> 1482119 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1482119 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1482119 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:44593_solr at time 18597453961702664
[junit4] 2> 1483120 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1483120 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1484120 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1484121 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1484121 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL firing registered processor for nodes: [127.0.0.1:44593_solr] added at times [18597453961702664], now=18597455963140581
[junit4] 2> 1484121 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"4212492d1d2d08T6bt1d5ayava1fix0yzd6i6n6d",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18597453961702664,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597453961702664],
[junit4] 2> "nodeNames":["127.0.0.1:44593_solr"]}}
[junit4] 2> 1484121 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_triggerL]
[junit4] 2> 1484122 INFO (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1484126 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"4212492d1d2d08T6bt1d5ayava1fix0yzd6i6n6d",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18597453961702664,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597453961702664],
[junit4] 2> "_enqueue_time_":18597455963870463,
[junit4] 2> "nodeNames":["127.0.0.1:44593_solr"]}}
[junit4] 2> 1484139 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerL after 5000ms
[junit4] 2> 1484139 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 13 ms for event id=4212492d1d2d08T6bt1d5ayava1fix0yzd6i6n6d
[junit4] 2> 1486137 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1486144 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1486144 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1486144 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1486145 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3a7933c5{/solr,null,AVAILABLE}
[junit4] 2> 1486145 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@46ccaa2a{SSL,[ssl, http/1.1]}{127.0.0.1:37923}
[junit4] 2> 1486145 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.Server Started @1487935ms
[junit4] 2> 1486145 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37923}
[junit4] 2> 1486146 ERROR (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1486146 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1486146 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1486146 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1486146 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T10:39:07.325Z
[junit4] 2> 1486155 INFO (zkConnectionManagerCallback-5019-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1486155 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1486178 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46415/solr
[junit4] 2> 1486183 INFO (zkConnectionManagerCallback-5023-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1486187 INFO (zkConnectionManagerCallback-5025-thread-1-processing-n:127.0.0.1:37923_solr) [n:127.0.0.1:37923_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1486194 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:37923_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1486195 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:37923_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1486197 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:37923_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37923_solr
[junit4] 2> 1486199 INFO (zkCallback-5014-thread-1-processing-n:127.0.0.1:44593_solr) [n:127.0.0.1:44593_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1486199 INFO (zkCallback-4994-thread-1-processing-n:127.0.0.1:33114_solr) [n:127.0.0.1:33114_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1486200 INFO (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1486200 INFO (zkCallback-5006-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1486220 INFO (zkCallback-5024-thread-1-processing-n:127.0.0.1:37923_solr) [n:127.0.0.1:37923_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1486279 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:37923_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37923.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1486292 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:37923_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37923.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1486292 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:37923_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37923.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1486293 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:37923_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_41AE5BAD79BAFD5-001/tempDir-001/node4/.
[junit4] 2> 1489139 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1489139 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1489139 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:37923_solr at time 18597460981660755
[junit4] 2> 1490140 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1490140 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1491140 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1491140 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1491140 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL firing registered processor for nodes: [127.0.0.1:37923_solr] added at times [18597460981660755], now=18597462982878056
[junit4] 2> 1491141 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"42124acf893c53T6bt1d5ayava1fix0yzd6i6n6f",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18597460981660755,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597460981660755],
[junit4] 2> "nodeNames":["127.0.0.1:37923_solr"]}}
[junit4] 2> 1491141 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_triggerL]
[junit4] 2> 1491142 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"42124acf893c53T6bt1d5ayava1fix0yzd6i6n6f",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18597460981660755,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597460981660755],
[junit4] 2> "_enqueue_time_":18597462983257769,
[junit4] 2> "nodeNames":["127.0.0.1:37923_solr"]}}
[junit4] 2> 1491142 WARN (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> java.lang.Exception: Error executing action: test1 for trigger event: {
[junit4] 2> "id":"42124acf893c53T6bt1d5ayava1fix0yzd6i6n6f",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":18597460981660755,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597460981660755],
[junit4] 2> "_enqueue_time_":18597462983257769,
[junit4] 2> "nodeNames":["127.0.0.1:37923_solr"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:302)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.RuntimeException: failure
[junit4] 2> at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction.process(TriggerIntegrationTest.java:989)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:299)
[junit4] 2> ... 6 more
[junit4] 2> 1491144 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerL after 5000ms
[junit4] 2> 1491144 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 2 ms for event id=42124acf893c53T6bt1d5ayava1fix0yzd6i6n6f
[junit4] 2> 1493142 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testListeners
[junit4] 2> 1493181 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testTriggerThrottling
[junit4] 2> 1493182 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 6
[junit4] 2> 1493182 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1493182 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1493183 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1493184 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1493184 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 1493188 INFO (qtp916929797-17708) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1493230 INFO (qtp1702494645-17797) [n:127.0.0.1:37923_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1493260 INFO (qtp19850307-17770) [n:127.0.0.1:44593_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1493263 INFO (qtp1614980409-17716) [n:127.0.0.1:46785_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1493263 DEBUG (qtp916929797-17704) [n:127.0.0.1:33114_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1493263 INFO (qtp916929797-17704) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=77
[junit4] 2> 1493264 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1493264 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37923_solr, 127.0.0.1:33114_solr, 127.0.0.1:46785_solr, 127.0.0.1:44593_solr]
[junit4] 2> 1493264 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1493264 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1493265 INFO (qtp916929797-17706) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1493267 INFO (qtp1702494645-17795) [n:127.0.0.1:37923_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1493271 INFO (qtp19850307-17771) [n:127.0.0.1:44593_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1493272 INFO (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1493272 INFO (qtp1614980409-17718) [n:127.0.0.1:46785_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1493273 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1493273 DEBUG (qtp916929797-17707) [n:127.0.0.1:33114_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1493273 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 1493273 INFO (qtp916929797-17707) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=9
[junit4] 2> 1493273 DEBUG (ScheduledTrigger-5303-thread-3) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1493273 DEBUG (ScheduledTrigger-5303-thread-3) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1493274 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 1493274 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37923_solr, 127.0.0.1:33114_solr, 127.0.0.1:46785_solr, 127.0.0.1:44593_solr]
[junit4] 2> 1493274 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1493274 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:37923_solr, 127.0.0.1:33114_solr, 127.0.0.1:46785_solr, 127.0.0.1:44593_solr]
[junit4] 2> 1493274 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1493274 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 1493275 INFO (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1493277 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T22:27:37+01:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1493277 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1493278 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 1493278 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1493278 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1493282 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1493282 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1493282 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1493290 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@451f0bed{/solr,null,AVAILABLE}
[junit4] 2> 1493290 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3bfa7e42{SSL,[ssl, http/1.1]}{127.0.0.1:50046}
[junit4] 2> 1493290 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.Server Started @1495080ms
[junit4] 2> 1493290 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50046}
[junit4] 2> 1493291 ERROR (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1493292 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1493293 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1493293 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1493293 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T10:39:14.472Z
[junit4] 2> 1493295 INFO (zkConnectionManagerCallback-5029-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1493296 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1493305 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46415/solr
[junit4] 2> 1493306 INFO (zkConnectionManagerCallback-5033-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1493309 INFO (zkConnectionManagerCallback-5035-thread-1-processing-n:127.0.0.1:50046_solr) [n:127.0.0.1:50046_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1493313 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:50046_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1493314 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:50046_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1493316 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:50046_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50046_solr
[junit4] 2> 1493317 INFO (zkCallback-4994-thread-1-processing-n:127.0.0.1:33114_solr) [n:127.0.0.1:33114_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1493317 INFO (zkCallback-5014-thread-1-processing-n:127.0.0.1:44593_solr) [n:127.0.0.1:44593_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1493317 INFO (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1493317 INFO (zkCallback-5024-thread-1-processing-n:127.0.0.1:37923_solr) [n:127.0.0.1:37923_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1493317 INFO (zkCallback-5006-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1493318 INFO (zkCallback-5034-thread-1-processing-n:127.0.0.1:50046_solr) [n:127.0.0.1:50046_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1493376 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:50046_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50046.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1493386 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:50046_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50046.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1493386 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:50046_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50046.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7e4ed970
[junit4] 2> 1493387 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [n:127.0.0.1:50046_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_41AE5BAD79BAFD5-001/tempDir-001/node5/.
[junit4] 2> 1494275 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1494275 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 5
[junit4] 2> 1494275 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:50046_solr at time 18597466117529098
[junit4] 2> 1494275 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:50046_solr] added at times [18597466117529098], now=18597466117576790
[junit4] 2> 1494275 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"42124c01a85e0aT6bt1d5ayava1fix0yzd6i6n6g",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":18597466117529098,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597466117529098],
[junit4] 2> "nodeNames":["127.0.0.1:50046_solr"]}}
[junit4] 2> 1494275 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger2, node_added_trigger1]
[junit4] 2> 1494276 INFO (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1494276 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"42124c01a85e0aT6bt1d5ayava1fix0yzd6i6n6g",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":18597466117529098,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597466117529098],
[junit4] 2> "_enqueue_time_":18597466117802674,
[junit4] 2> "nodeNames":["127.0.0.1:50046_solr"]}}
[junit4] 2> 1494276 INFO (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_added_trigger1
[junit4] 2> 1494277 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 1494277 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 1494277 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=42124c01a85e0aT6bt1d5ayava1fix0yzd6i6n6g
[junit4] 2> 1499277 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1499277 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 5
[junit4] 2> 1499277 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:50046_solr at time 18597471119753981
[junit4] 2> 1499277 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 firing registered processor for nodes: [127.0.0.1:50046_solr] added at times [18597471119753981], now=18597471119798113
[junit4] 2> 1499277 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"42124d2bd042fdT6bt1d5ayava1fix0yzd6i6n6i",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":18597471119753981,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597471119753981],
[junit4] 2> "nodeNames":["127.0.0.1:50046_solr"]}}
[junit4] 2> 1499277 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_trigger2, node_added_trigger1]
[junit4] 2> 1499278 INFO (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1499278 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"42124d2bd042fdT6bt1d5ayava1fix0yzd6i6n6i",
[junit4] 2> "source":"node_added_trigger2",
[junit4] 2> "eventTime":18597471119753981,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597471119753981],
[junit4] 2> "_enqueue_time_":18597471119997625,
[junit4] 2> "nodeNames":["127.0.0.1:50046_solr"]}}
[junit4] 2> 1499278 INFO (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.TriggerIntegrationTest last action at 1518345555455000000 current time = 1518345560457000000
[junit4] 2> real diff: 5002000000
[junit4] 2> min diff: 4998000000
[junit4] 2> 1499278 INFO (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_added_trigger2
[junit4] 2> 1499279 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 1499279 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 1499279 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 1 ms for event id=42124d2bd042fdT6bt1d5ayava1fix0yzd6i6n6i
[junit4] 2> 1499283 INFO (qtp916929797-17708) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1499285 INFO (qtp1702494645-17797) [n:127.0.0.1:37923_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1499287 INFO (qtp19850307-17772) [n:127.0.0.1:44593_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1499289 INFO (qtp1614980409-17716) [n:127.0.0.1:46785_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1499305 INFO (qtp621424185-17820) [n:127.0.0.1:50046_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1499306 DEBUG (qtp916929797-17704) [n:127.0.0.1:33114_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1499306 INFO (qtp916929797-17704) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=25
[junit4] 2> 1499306 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 1499307 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:50046_solr, 127.0.0.1:37923_solr, 127.0.0.1:33114_solr, 127.0.0.1:46785_solr, 127.0.0.1:44593_solr]
[junit4] 2> 1499307 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1499307 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:50046_solr, 127.0.0.1:37923_solr, 127.0.0.1:33114_solr, 127.0.0.1:46785_solr, 127.0.0.1:44593_solr]
[junit4] 2> 1499307 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:50046_solr, 127.0.0.1:37923_solr, 127.0.0.1:33114_solr, 127.0.0.1:46785_solr, 127.0.0.1:44593_solr]
[junit4] 2> 1499307 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1499307 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 1499308 INFO (qtp916929797-17706) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1499309 INFO (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1499309 INFO (qtp1702494645-17795) [n:127.0.0.1:37923_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1499309 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 1499310 DEBUG (ScheduledTrigger-5303-thread-3) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 5
[junit4] 2> 1499310 INFO (qtp19850307-17771) [n:127.0.0.1:44593_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1499311 INFO (qtp1614980409-17718) [n:127.0.0.1:46785_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1499313 INFO (qtp621424185-17822) [n:127.0.0.1:50046_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1499313 DEBUG (qtp916929797-17707) [n:127.0.0.1:33114_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1499313 INFO (qtp916929797-17707) [n:127.0.0.1:33114_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 1499313 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 1499314 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:50046_solr, 127.0.0.1:37923_solr, 127.0.0.1:33114_solr, 127.0.0.1:46785_solr, 127.0.0.1:44593_solr]
[junit4] 2> 1499314 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger2 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1499314 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:50046_solr, 127.0.0.1:37923_solr, 127.0.0.1:33114_solr, 127.0.0.1:46785_solr, 127.0.0.1:44593_solr]
[junit4] 2> 1499314 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:50046_solr, 127.0.0.1:37923_solr, 127.0.0.1:33114_solr, 127.0.0.1:46785_solr, 127.0.0.1:44593_solr]
[junit4] 2> 1499314 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$ThrottlingTesterAction}], enabled=true}
[junit4] 2> 1499314 DEBUG (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:50046_solr, 127.0.0.1:37923_solr, 127.0.0.1:33114_solr, 127.0.0.1:46785_solr, 127.0.0.1:44593_solr]
[junit4] 2> 1499314 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 1499315 INFO (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1499316 DEBUG (OverseerAutoScalingTriggerThread-73276396651216902-127.0.0.1:46785_solr-n_0000000000) [n:127.0.0.1:46785_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 1499316 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 5
[junit4] 2> 1499316 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@3bfa7e42{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1499316 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=73993072
[junit4] 2> 1499317 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1499317 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6a718ba7: rootName = solr_50046, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@33933ce1
[junit4] 2> 1499323 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1499323 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@751da017: rootName = solr_50046, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@49c9a4c4
[junit4] 2> 1499327 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1499327 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5da41655: rootName = solr_50046, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@26cf07f2
[junit4] 2> 1499331 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:50046_solr
[junit4] 2> 1499332 INFO (zkCallback-4994-thread-1-processing-n:127.0.0.1:33114_solr) [n:127.0.0.1:33114_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 1499332 INFO (zkCallback-5024-thread-1-processing-n:127.0.0.1:37923_solr) [n:127.0.0.1:37923_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 1499332 INFO (zkCallback-5014-thread-1-processing-n:127.0.0.1:44593_solr) [n:127.0.0.1:44593_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 1499332 INFO (zkCallback-5006-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 1499333 INFO (zkCallback-4996-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 1499333 INFO (zkCallback-5034-thread-1-processing-n:127.0.0.1:50046_solr) [n:127.0.0.1:50046_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 1499333 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1499333 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1499335 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@451f0bed{/solr,null,UNAVAILABLE}
[junit4] 2> 1499335 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1500311 DEBUG (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 4
[junit4] 2> 1500311 DEBUG (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:50046_solr
[junit4] 2> 1500311 DEBUG (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:50046_solr]
[junit4] 2> 1500311 DEBUG (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"42124d696c1dddT6bt1d5ayava1fix0yzd6i6n6k",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":18597472153378269,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597472153378269],
[junit4] 2> "nodeNames":["127.0.0.1:50046_solr"]}}
[junit4] 2> 1500311 DEBUG (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 1500312 INFO (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1500312 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"42124d696c1dddT6bt1d5ayava1fix0yzd6i6n6k",
[junit4] 2> "source":"node_lost_trigger1",
[junit4] 2> "eventTime":18597472153378269,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597472153378269],
[junit4] 2> "_enqueue_time_":18597472153579318,
[junit4] 2> "nodeNames":["127.0.0.1:50046_solr"]}}
[junit4] 2> 1500312 INFO (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_lost_trigger1
[junit4] 2> 1500312 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 5000ms
[junit4] 2> 1500312 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 1500313 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 5000ms
[junit4] 2> 1500313 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 1500313 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 0 ms for event id=42124d696c1dddT6bt1d5ayava1fix0yzd6i6n6k
[junit4] 2> 1505313 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger2
[junit4] 2> 1505313 DEBUG (ScheduledTrigger-5303-thread-2) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1505313 DEBUG (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger1 with currently live nodes: 4
[junit4] 2> 1505313 DEBUG (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1505313 DEBUG (ScheduledTrigger-5303-thread-1) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 1505316 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger2 with currently live nodes: 4
[junit4] 2> 1505317 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:50046_solr
[junit4] 2> 1505317 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:50046_solr]
[junit4] 2> 1505317 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"42124e93c9d7b0T6bt1d5ayava1fix0yzd6i6n6m",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":18597477159131056,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597477159131056],
[junit4] 2> "nodeNames":["127.0.0.1:50046_solr"]}}
[junit4] 2> 1505317 DEBUG (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_trigger2, node_added_trigger2, node_lost_trigger1, node_added_trigger1]
[junit4] 2> 1505318 INFO (ScheduledTrigger-5303-thread-4) [n:127.0.0.1:46785_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1505318 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"42124e93c9d7b0T6bt1d5ayava1fix0yzd6i6n6m",
[junit4] 2> "source":"node_lost_trigger2",
[junit4] 2> "eventTime":18597477159131056,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[18597477159131056],
[junit4] 2> "_enqueue_time_":18597477159504953,
[junit4] 2> "nodeNames":["127.0.0.1:50046_solr"]}}
[junit4] 2> 1505318 INFO (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.TriggerIntegrationTest last action at 1518345561491000000 current time = 1518345566497000000
[junit4] 2> real diff: 5006000000
[junit4] 2> min diff: 4998000000
[junit4] 2> 1505318 INFO (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.TriggerIntegrationTest action executed from node_lost_trigger2
[junit4] 2> 1505318 INFO (TEST-TriggerIntegrationTest.testTriggerThrottling-seed#[41AE5BAD79BAFD5]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testTriggerThrottling
[junit4] 2> 1505322 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger2 after 5000ms
[junit4] 2> 1505322 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger2 after 5000ms
[junit4] 2> 1505322 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger1 after 5000ms
[junit4] 2> 1505322 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 5000ms
[junit4] 2> 1505322 DEBUG (AutoscalingActionExecutor-5304-thread-1-processing-n:127.0.0.1:46785_solr) [n:127.0.0.1:46785_solr ] o.a.s.c.
[...truncated too long message...]
47844, domain = solr.core.testMetricTrigger.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.testMetricTrigger.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@1470c27d
[junit4] 2> 1679032 INFO (jetty-closer-4979-thread-6) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4f8b03e6{/solr,null,UNAVAILABLE}
[junit4] 2> 1679033 INFO (jetty-closer-4979-thread-6) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1679039 INFO (coreCloseExecutor-5531-thread-1) [n:127.0.0.1:42056_solr c:testMetricTrigger s:shard1 r:core_node8 x:testMetricTrigger_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testMetricTrigger.shard1.leader, tag=1313771692
[junit4] 2> 1679040 INFO (jetty-closer-4979-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@55c5dff3{/solr,null,UNAVAILABLE}
[junit4] 2> 1679041 INFO (jetty-closer-4979-thread-4) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1679042 INFO (coreCloseExecutor-5521-thread-3) [n:127.0.0.1:47844_solr c:testMetricTrigger s:shard2 r:core_node4 x:testMetricTrigger_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.testMetricTrigger.shard2.leader, tag=2018443833
[junit4] 2> 1679042 INFO (coreCloseExecutor-5521-thread-2) [n:127.0.0.1:47844_solr c:collection1 s:shard1 r:core_node8 x:collection1_shard1_replica_n7] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=987913674
[junit4] 2> 1679044 INFO (jetty-closer-4979-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73276396651216919-127.0.0.1:47844_solr-n_0000000006) closing
[junit4] 2> 1679045 INFO (OverseerStateUpdate-73276396651216919-127.0.0.1:47844_solr-n_0000000006) [n:127.0.0.1:47844_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:47844_solr
[junit4] 2> 1679045 WARN (OverseerAutoScalingTriggerThread-73276396651216919-127.0.0.1:47844_solr-n_0000000006) [n:127.0.0.1:47844_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1679047 DEBUG (jetty-closer-4979-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1680431 WARN (zkCallback-5134-thread-1-processing-n:127.0.0.1:52662_solr) [n:127.0.0.1:52662_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1680432 INFO (jetty-closer-4979-thread-8) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2ba22464{/solr,null,UNAVAILABLE}
[junit4] 2> 1680432 INFO (jetty-closer-4979-thread-8) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1680433 INFO (jetty-closer-4979-thread-10) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@14cb12f6{/solr,null,UNAVAILABLE}
[junit4] 2> 1680433 INFO (jetty-closer-4979-thread-10) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1680548 INFO (jetty-closer-4979-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2527492c{/solr,null,UNAVAILABLE}
[junit4] 2> 1680548 INFO (jetty-closer-4979-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1680549 ERROR (SUITE-TriggerIntegrationTest-seed#[41AE5BAD79BAFD5]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1680549 INFO (SUITE-TriggerIntegrationTest-seed#[41AE5BAD79BAFD5]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46415 46415
[junit4] 2> 1680633 INFO (Thread-2517) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46415 46415
[junit4] 2> 1680633 WARN (Thread-2517) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 39 /solr/collections/collection1/terms/shard1
[junit4] 2> 22 /solr/aliases.json
[junit4] 2> 21 /solr/security.json
[junit4] 2> 10 /solr/configs/conf
[junit4] 2> 9 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 137 /solr/collections/collection1/state.json
[junit4] 2> 40 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 22 /solr/clusterprops.json
[junit4] 2> 22 /solr/clusterstate.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 3 /solr/overseer_elect/election/73276396651216950-127.0.0.1:33416_solr-n_0000000016
[junit4] 2> 2 /solr/overseer_elect/election/73276396651216934-127.0.0.1:37023_solr-n_0000000011
[junit4] 2> 2 /solr/overseer_elect/election/73276396651216928-127.0.0.1:42056_solr-n_0000000009
[junit4] 2> 2 /solr/overseer_elect/election/73276396651216937-127.0.0.1:33478_solr-n_0000000012
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73276396651216901-core_node12-n_0000000005
[junit4] 2> 2 /solr/overseer_elect/election/73276396651216941-127.0.0.1:52662_solr-n_0000000013
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73276396651216922-core_node10-n_0000000004
[junit4] 2> 2 /solr/overseer_elect/election/73276396651216910-127.0.0.1:37923_solr-n_0000000003
[junit4] 2> 2 /solr/overseer_elect/election/73276396651216922-127.0.0.1:47310_solr-n_0000000007
[junit4] 2> 2 /solr/overseer_elect/election/73276396651216919-127.0.0.1:47844_solr-n_0000000006
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 22 /solr/live_nodes
[junit4] 2> 22 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/queue
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_41AE5BAD79BAFD5-001
[junit4] 2> Feb 11, 2018 10:42:21 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=LuceneVarGapDocFreqInterval), x_s=Lucene50(blocksize=128)}, docValues:{}, maxPointsInLeafNode=855, maxMBSortInHeap=5.338215179979864, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@24b3d39c), locale=he-IL, timezone=America/Manaus
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=128739976,total=505413632
[junit4] 2> NOTE: All tests run in this JVM: [TestReplicationHandlerBackup, TestFiltering, DocValuesMultiTest, TestInPlaceUpdatesStandalone, BufferStoreTest, TestStressReorder, TestEmbeddedSolrServerConstructors, UnloadDistributedZkTest, BaseCdcrDistributedZkTest, ClusterStateTest, SparseHLLTest, CdcrVersionReplicationTest, NoCacheHeaderTest, TestExclusionRuleCollectionAccess, SolrMetricsIntegrationTest, TestReplicaProperties, TestCollectionAPIs, TestConfigReload, BasicAuthStandaloneTest, SegmentsInfoRequestHandlerTest, TestFieldCacheSort, ConvertedLegacyTest, TestBlendedInfixSuggestions, DistributedSpellCheckComponentTest, TestSlowCompositeReaderWrapper, TestLFUCache, TestFieldCollectionResource, SuggesterTSTTest, ConjunctionSolrSpellCheckerTest, TestGraphTermsQParserPlugin, TestCloudNestedDocsSort, ResponseLogComponentTest, TestLazyCores, IndexSchemaRuntimeFieldTest, BlockJoinFacetRandomTest, TestSubQueryTransformerCrossCore, ReplaceNodeNoTargetTest, TestLegacyNumericUtils, AnalysisAfterCoreReloadTest, QueryResultKeyTest, TestNRTOpen, SimpleMLTQParserTest, DistributedFacetExistsSmallTest, TestCopyFieldCollectionResource, TestRebalanceLeaders, TestLargeCluster, CursorMarkTest, ShowFileRequestHandlerTest, TestPayloadCheckQParserPlugin, MultiTermTest, TestQuerySenderListener, DirectSolrSpellCheckerTest, SpatialRPTFieldTypeTest, HdfsNNFailoverTest, StressHdfsTest, AliasIntegrationTest, TestExceedMaxTermLength, ClusterStateUpdateTest, TestDFRSimilarityFactory, SolrCloudReportersTest, TimeZoneUtilsTest, TestScoreJoinQPScore, SolrTestCaseJ4Test, TestCollectionsAPIViaSolrCloudCluster, ConnectionReuseTest, ShardSplitTest, TestSolrCoreParser, ExecutePlanActionTest, URLClassifyProcessorTest, OverriddenZkACLAndCredentialsProvidersTest, RestartWhileUpdatingTest, TestFastWriter, DocValuesTest, DirectoryFactoryTest, TestConfigSetsAPIExclusivity, TestSolrConfigHandlerConcurrent, TestFieldCacheWithThreads, DeleteInactiveReplicaTest, TestRandomDVFaceting, TestImplicitCoreProperties, DistanceFunctionTest, TestFieldTypeResource, TestCoreContainer, CleanupOldIndexTest, TestExactStatsCache, LeaderFailoverAfterPartitionTest, TestQueryTypes, JavabinLoaderTest, TriLevelCompositeIdRoutingTest, ZkShardTermsTest, LeaderElectionIntegrationTest, TestPullReplicaErrorHandling, PreAnalyzedFieldTest, TestSortableTextField, TestStressLiveNodes, SolrJmxReporterCloudTest, RecoveryZkTest, TestCustomDocTransformer, CollectionsAPISolrJTest, TestSimpleQParserPlugin, AnalysisErrorHandlingTest, TestHighFrequencyDictionaryFactory, TestManagedSynonymGraphFilterFactory, TriggerIntegrationTest]
[junit4] Completed [390/776 (1!)] on J2 in 201.10s, 14 tests, 1 failure <<< FAILURES!
[...truncated 49585 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2309 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2309/
2 tests failed.
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Could not find collection:collection2
Stack Trace:
java.lang.AssertionError: Could not find collection:collection2
at __randomizedtesting.SeedInfo.seed([76F376A29D9976EF:FEA7497833651B17]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:915)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:612)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ReplaceNodeNoTargetTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([76F376A29D9976EF:FEA7497833651B17]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12670 lines...]
[junit4] Suite: org.apache.solr.cloud.ReplaceNodeNoTargetTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/init-core-data-001
[junit4] 2> 673366 WARN (SUITE-ReplaceNodeNoTargetTest-seed#[76F376A29D9976EF]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=9 numCloses=9
[junit4] 2> 673366 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[76F376A29D9976EF]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 673368 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[76F376A29D9976EF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 673368 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[76F376A29D9976EF]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 673368 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[76F376A29D9976EF]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 6 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001
[junit4] 2> 673368 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[76F376A29D9976EF]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 673369 INFO (Thread-1167) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 673369 INFO (Thread-1167) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 673371 ERROR (Thread-1167) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 673469 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[76F376A29D9976EF]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:56397
[junit4] 2> 673470 INFO (zkConnectionManagerCallback-2765-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673473 INFO (jetty-launcher-2762-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T18:27:37-03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 673473 INFO (jetty-launcher-2762-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T18:27:37-03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 673473 INFO (jetty-launcher-2762-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T18:27:37-03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 673474 INFO (jetty-launcher-2762-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T18:27:37-03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 673474 INFO (jetty-launcher-2762-thread-6) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T18:27:37-03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 673475 INFO (jetty-launcher-2762-thread-5) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T18:27:37-03:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 673476 INFO (jetty-launcher-2762-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 673476 INFO (jetty-launcher-2762-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 673476 INFO (jetty-launcher-2762-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 673476 INFO (jetty-launcher-2762-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 673476 INFO (jetty-launcher-2762-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 673477 INFO (jetty-launcher-2762-thread-4) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 673477 INFO (jetty-launcher-2762-thread-6) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 673477 INFO (jetty-launcher-2762-thread-6) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 673477 INFO (jetty-launcher-2762-thread-6) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 673477 INFO (jetty-launcher-2762-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6aaa2df1{/solr,null,AVAILABLE}
[junit4] 2> 673477 INFO (jetty-launcher-2762-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@68c471d4{/solr,null,AVAILABLE}
[junit4] 2> 673477 INFO (jetty-launcher-2762-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 673477 INFO (jetty-launcher-2762-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 673477 INFO (jetty-launcher-2762-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 673478 INFO (jetty-launcher-2762-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 673478 INFO (jetty-launcher-2762-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 673478 INFO (jetty-launcher-2762-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 673478 INFO (jetty-launcher-2762-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3e2ea3f0{/solr,null,AVAILABLE}
[junit4] 2> 673479 INFO (jetty-launcher-2762-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@794be3f9{HTTP/1.1,[http/1.1]}{127.0.0.1:?????}
[junit4] 2> 673479 INFO (jetty-launcher-2762-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3664a034{/solr,null,AVAILABLE}
[junit4] 2> 673479 INFO (jetty-launcher-2762-thread-3) [ ] o.e.j.s.Server Started @??????ms
[junit4] 2> 673479 INFO (jetty-launcher-2762-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36661}
[junit4] 2> 673480 INFO (jetty-launcher-2762-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@13b04a13{HTTP/1.1,[http/1.1]}{127.0.0.1:?????}
[junit4] 2> 673480 ERROR (jetty-launcher-2762-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 673480 INFO (jetty-launcher-2762-thread-4) [ ] o.e.j.s.Server Started @??????ms
[junit4] 2> 673480 INFO (jetty-launcher-2762-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 673480 INFO (jetty-launcher-2762-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33961}
[junit4] 2> 673480 INFO (jetty-launcher-2762-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 673480 INFO (jetty-launcher-2762-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 673480 INFO (jetty-launcher-2762-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T08:29:46.876Z
[junit4] 2> 673480 ERROR (jetty-launcher-2762-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 673480 INFO (jetty-launcher-2762-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 673480 INFO (jetty-launcher-2762-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 673480 INFO (jetty-launcher-2762-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 673480 INFO (jetty-launcher-2762-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T08:29:46.876Z
[junit4] 2> 673481 INFO (jetty-launcher-2762-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7de820ff{HTTP/1.1,[http/1.1]}{127.0.0.1:?????}
[junit4] 2> 673481 INFO (jetty-launcher-2762-thread-5) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 673481 INFO (jetty-launcher-2762-thread-1) [ ] o.e.j.s.Server Started @??????ms
[junit4] 2> 673481 INFO (jetty-launcher-2762-thread-5) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 673481 INFO (jetty-launcher-2762-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=48615}
[junit4] 2> 673481 INFO (jetty-launcher-2762-thread-5) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 673481 INFO (jetty-launcher-2762-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2c451fb1{HTTP/1.1,[http/1.1]}{127.0.0.1:?????}
[junit4] 2> 673481 INFO (jetty-launcher-2762-thread-2) [ ] o.e.j.s.Server Started @??????ms
[junit4] 2> 673481 INFO (jetty-launcher-2762-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=32996}
[junit4] 2> 673481 ERROR (jetty-launcher-2762-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 673481 INFO (jetty-launcher-2762-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5c002fa5{/solr,null,AVAILABLE}
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 673482 ERROR (jetty-launcher-2762-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T08:29:46.878Z
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-5) [ ] o.e.j.s.AbstractConnector Started ServerConnector@764898a3{HTTP/1.1,[http/1.1]}{127.0.0.1:?????}
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-5) [ ] o.e.j.s.Server Started @??????ms
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T08:29:46.878Z
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=56776}
[junit4] 2> 673482 ERROR (jetty-launcher-2762-thread-5) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-5) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-5) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-5) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 673482 INFO (jetty-launcher-2762-thread-5) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T08:29:46.878Z
[junit4] 2> 673485 INFO (jetty-launcher-2762-thread-6) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@65e6a5f9{/solr,null,AVAILABLE}
[junit4] 2> 673485 INFO (jetty-launcher-2762-thread-6) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2a7a96f9{HTTP/1.1,[http/1.1]}{127.0.0.1:?????}
[junit4] 2> 673485 INFO (jetty-launcher-2762-thread-6) [ ] o.e.j.s.Server Started @??????ms
[junit4] 2> 673485 INFO (jetty-launcher-2762-thread-6) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38116}
[junit4] 2> 673485 ERROR (jetty-launcher-2762-thread-6) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 673486 INFO (jetty-launcher-2762-thread-6) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 673486 INFO (jetty-launcher-2762-thread-6) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 673486 INFO (jetty-launcher-2762-thread-6) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 673486 INFO (jetty-launcher-2762-thread-6) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T08:29:46.882Z
[junit4] 2> 673489 INFO (zkConnectionManagerCallback-2767-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673493 INFO (jetty-launcher-2762-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 673493 INFO (zkConnectionManagerCallback-2771-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673494 INFO (jetty-launcher-2762-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 673496 INFO (zkConnectionManagerCallback-2777-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673496 INFO (zkConnectionManagerCallback-2773-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673497 INFO (jetty-launcher-2762-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 673497 INFO (jetty-launcher-2762-thread-6) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 673504 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x104540845950001, likely client has closed socket
[junit4] 2> 673504 INFO (zkConnectionManagerCallback-2769-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673504 INFO (zkConnectionManagerCallback-2775-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673505 INFO (jetty-launcher-2762-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 673507 INFO (jetty-launcher-2762-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 673510 INFO (jetty-launcher-2762-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56397/solr
[junit4] 2> 673512 INFO (jetty-launcher-2762-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56397/solr
[junit4] 2> 673514 INFO (jetty-launcher-2762-thread-6) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56397/solr
[junit4] 2> 673515 INFO (jetty-launcher-2762-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56397/solr
[junit4] 2> 673515 INFO (zkConnectionManagerCallback-2781-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673515 INFO (jetty-launcher-2762-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56397/solr
[junit4] 2> 673515 INFO (zkConnectionManagerCallback-2785-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673515 INFO (jetty-launcher-2762-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:56397/solr
[junit4] 2> 673518 INFO (zkConnectionManagerCallback-2795-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673519 INFO (zkConnectionManagerCallback-2797-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673519 INFO (zkConnectionManagerCallback-2801-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673521 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x104540845950008, likely client has closed socket
[junit4] 2> 673521 INFO (zkConnectionManagerCallback-2803-thread-1-processing-n:127.0.0.1:36661_solr) [n:127.0.0.1:36661_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673522 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x104540845950009, likely client has closed socket
[junit4] 2> 673522 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10454084595000a, likely client has closed socket
[junit4] 2> 673522 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10454084595000b, likely client has closed socket
[junit4] 2> 673522 INFO (zkConnectionManagerCallback-2799-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673523 INFO (zkConnectionManagerCallback-2805-thread-1-processing-n:127.0.0.1:48615_solr) [n:127.0.0.1:48615_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673524 INFO (zkConnectionManagerCallback-2811-thread-1-processing-n:127.0.0.1:38116_solr) [n:127.0.0.1:38116_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673524 INFO (zkConnectionManagerCallback-2807-thread-1-processing-n:127.0.0.1:33961_solr) [n:127.0.0.1:33961_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673525 INFO (zkConnectionManagerCallback-2809-thread-1-processing-n:127.0.0.1:56776_solr) [n:127.0.0.1:56776_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673536 INFO (zkConnectionManagerCallback-2813-thread-1-processing-n:127.0.0.1:32996_solr) [n:127.0.0.1:32996_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 673549 INFO (jetty-launcher-2762-thread-2) [n:127.0.0.1:32996_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 673550 INFO (jetty-launcher-2762-thread-2) [n:127.0.0.1:32996_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:32996_solr
[junit4] 2> 673551 INFO (jetty-launcher-2762-thread-2) [n:127.0.0.1:32996_solr ] o.a.s.c.Overseer Overseer (id=73275888448634898-127.0.0.1:32996_solr-n_0000000000) starting
[junit4] 2> 673561 INFO (jetty-launcher-2762-thread-2) [n:127.0.0.1:32996_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32996_solr
[junit4] 2> 673573 INFO (OverseerStateUpdate-73275888448634898-127.0.0.1:32996_solr-n_0000000000) [n:127.0.0.1:32996_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 673610 INFO (jetty-launcher-2762-thread-6) [n:127.0.0.1:38116_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 673613 INFO (jetty-launcher-2762-thread-6) [n:127.0.0.1:38116_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 673616 INFO (jetty-launcher-2762-thread-4) [n:127.0.0.1:33961_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 673616 INFO (jetty-launcher-2762-thread-6) [n:127.0.0.1:38116_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38116_solr
[junit4] 2> 673622 INFO (zkCallback-2812-thread-1-processing-n:127.0.0.1:32996_solr) [n:127.0.0.1:32996_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 673627 INFO (jetty-launcher-2762-thread-4) [n:127.0.0.1:33961_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 673631 INFO (jetty-launcher-2762-thread-4) [n:127.0.0.1:33961_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33961_solr
[junit4] 2> 673641 INFO (zkCallback-2810-thread-1-processing-n:127.0.0.1:38116_solr) [n:127.0.0.1:38116_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 673645 INFO (zkCallback-2812-thread-1-processing-n:127.0.0.1:32996_solr) [n:127.0.0.1:32996_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 673649 INFO (zkCallback-2806-thread-1-processing-n:127.0.0.1:33961_solr) [n:127.0.0.1:33961_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 673649 INFO (zkCallback-2810-thread-2-processing-n:127.0.0.1:38116_solr) [n:127.0.0.1:38116_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 673655 INFO (jetty-launcher-2762-thread-3) [n:127.0.0.1:36661_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 673658 INFO (jetty-launcher-2762-thread-3) [n:127.0.0.1:36661_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 673660 INFO (jetty-launcher-2762-thread-3) [n:127.0.0.1:36661_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36661_solr
[junit4] 2> 673660 INFO (jetty-launcher-2762-thread-1) [n:127.0.0.1:48615_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 673669 INFO (zkCallback-2810-thread-2-processing-n:127.0.0.1:38116_solr) [n:127.0.0.1:38116_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 673669 INFO (zkCallback-2806-thread-1-processing-n:127.0.0.1:33961_solr) [n:127.0.0.1:33961_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 673669 INFO (zkCallback-2812-thread-1-processing-n:127.0.0.1:32996_solr) [n:127.0.0.1:32996_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 673676 INFO (zkCallback-2802-thread-1-processing-n:127.0.0.1:36661_solr) [n:127.0.0.1:36661_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 673685 INFO (zkCallback-2804-thread-1-processing-n:127.0.0.1:48615_solr) [n:127.0.0.1:48615_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 673705 INFO (jetty-launcher-2762-thread-1) [n:127.0.0.1:48615_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 673709 INFO (jetty-launcher-2762-thread-1) [n:127.0.0.1:48615_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:48615_solr
[junit4] 2> 673711 INFO (zkCallback-2810-thread-2-processing-n:127.0.0.1:38116_solr) [n:127.0.0.1:38116_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 673711 INFO (zkCallback-2812-thread-1-processing-n:127.0.0.1:32996_solr) [n:127.0.0.1:32996_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 673711 INFO (zkCallback-2802-thread-1-processing-n:127.0.0.1:36661_solr) [n:127.0.0.1:36661_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 673711 INFO (zkCallback-2806-thread-1-processing-n:127.0.0.1:33961_solr) [n:127.0.0.1:33961_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 673713 INFO (zkCallback-2804-thread-1-processing-n:127.0.0.1:48615_solr) [n:127.0.0.1:48615_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 673724 INFO (jetty-launcher-2762-thread-5) [n:127.0.0.1:56776_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 673725 INFO (jetty-launcher-2762-thread-5) [n:127.0.0.1:56776_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 673727 INFO (jetty-launcher-2762-thread-5) [n:127.0.0.1:56776_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56776_solr
[junit4] 2> 673735 INFO (zkCallback-2810-thread-2-processing-n:127.0.0.1:38116_solr) [n:127.0.0.1:38116_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 673735 INFO (zkCallback-2812-thread-1-processing-n:127.0.0.1:32996_solr) [n:127.0.0.1:32996_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 673735 INFO (zkCallback-2802-thread-1-processing-n:127.0.0.1:36661_solr) [n:127.0.0.1:36661_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 673735 INFO (zkCallback-2806-thread-1-processing-n:127.0.0.1:33961_solr) [n:127.0.0.1:33961_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 673735 INFO (zkCallback-2804-thread-1-processing-n:127.0.0.1:48615_solr) [n:127.0.0.1:48615_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 673736 INFO (jetty-launcher-2762-thread-6) [n:127.0.0.1:38116_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38116.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673743 INFO (jetty-launcher-2762-thread-6) [n:127.0.0.1:38116_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38116.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673744 INFO (jetty-launcher-2762-thread-6) [n:127.0.0.1:38116_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38116.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673744 INFO (jetty-launcher-2762-thread-6) [n:127.0.0.1:38116_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node5/.
[junit4] 2> 673760 INFO (jetty-launcher-2762-thread-2) [n:127.0.0.1:32996_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32996.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673766 INFO (jetty-launcher-2762-thread-2) [n:127.0.0.1:32996_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32996.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673766 INFO (jetty-launcher-2762-thread-2) [n:127.0.0.1:32996_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32996.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673767 INFO (jetty-launcher-2762-thread-2) [n:127.0.0.1:32996_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node2/.
[junit4] 2> 673785 INFO (zkCallback-2808-thread-1-processing-n:127.0.0.1:56776_solr) [n:127.0.0.1:56776_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (6)
[junit4] 2> 673851 INFO (jetty-launcher-2762-thread-1) [n:127.0.0.1:48615_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48615.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673851 INFO (jetty-launcher-2762-thread-5) [n:127.0.0.1:56776_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56776.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673853 INFO (jetty-launcher-2762-thread-3) [n:127.0.0.1:36661_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36661.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673893 INFO (jetty-launcher-2762-thread-1) [n:127.0.0.1:48615_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48615.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673893 INFO (jetty-launcher-2762-thread-1) [n:127.0.0.1:48615_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48615.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673894 INFO (jetty-launcher-2762-thread-5) [n:127.0.0.1:56776_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56776.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673894 INFO (jetty-launcher-2762-thread-5) [n:127.0.0.1:56776_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56776.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673896 INFO (jetty-launcher-2762-thread-3) [n:127.0.0.1:36661_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36661.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673896 INFO (jetty-launcher-2762-thread-3) [n:127.0.0.1:36661_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36661.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673897 INFO (jetty-launcher-2762-thread-1) [n:127.0.0.1:48615_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node1/.
[junit4] 2> 673898 INFO (jetty-launcher-2762-thread-5) [n:127.0.0.1:56776_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node6/.
[junit4] 2> 673898 INFO (jetty-launcher-2762-thread-4) [n:127.0.0.1:33961_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33961.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673898 INFO (jetty-launcher-2762-thread-3) [n:127.0.0.1:36661_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node3/.
[junit4] 2> 673928 INFO (jetty-launcher-2762-thread-4) [n:127.0.0.1:33961_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33961.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673928 INFO (jetty-launcher-2762-thread-4) [n:127.0.0.1:33961_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33961.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 673929 INFO (jetty-launcher-2762-thread-4) [n:127.0.0.1:33961_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node4/.
[junit4] 2> 674102 INFO (zkConnectionManagerCallback-2827-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 674105 INFO (zkConnectionManagerCallback-2831-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 674106 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[76F376A29D9976EF]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (6)
[junit4] 2> 674106 INFO (SUITE-ReplaceNodeNoTargetTest-seed#[76F376A29D9976EF]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:56397/solr ready
[junit4] 2> 674149 INFO (TEST-ReplaceNodeNoTargetTest.test-seed#[76F376A29D9976EF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 674150 INFO (zkConnectionManagerCallback-2833-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 674150 INFO (TEST-ReplaceNodeNoTargetTest.test-seed#[76F376A29D9976EF]) [ ] o.a.s.c.ReplaceNodeNoTargetTest total_jettys: 6
[junit4] 2> 674164 INFO (qtp1329496417-6475) [n:127.0.0.1:32996_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674165 INFO (qtp1976465646-6483) [n:127.0.0.1:33961_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674167 INFO (qtp152333974-6477) [n:127.0.0.1:36661_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674168 INFO (qtp342457549-6479) [n:127.0.0.1:38116_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674171 INFO (qtp1266990277-6467) [n:127.0.0.1:48615_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674172 INFO (qtp727093809-6517) [n:127.0.0.1:56776_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674174 INFO (qtp727093809-6482) [n:127.0.0.1:56776_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=20
[junit4] 2> 674175 INFO (qtp727093809-6514) [n:127.0.0.1:56776_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=2&collection.configName=conf1&name=replacenodetest_coll_notarget&nrtReplicas=2&action=CREATE&numShards=5&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 674176 INFO (OverseerThreadFactory-1538-thread-1-processing-n:127.0.0.1:32996_solr) [n:127.0.0.1:32996_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection replacenodetest_coll_notarget
[junit4] 2> 674183 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674190 INFO (qtp1976465646-6496) [n:127.0.0.1:33961_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674191 INFO (qtp152333974-6508) [n:127.0.0.1:36661_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674197 INFO (qtp342457549-6499) [n:127.0.0.1:38116_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674199 INFO (qtp1266990277-6476) [n:127.0.0.1:48615_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674200 INFO (qtp727093809-6516) [n:127.0.0.1:56776_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 674211 INFO (OverseerThreadFactory-1538-thread-1-processing-n:127.0.0.1:32996_solr) [n:127.0.0.1:32996_solr ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 18589698270 sessionWrapper.createTime 18589698259310811, this.sessionWrapper.createTime 18589698259310811
[junit4] 2> 674317 INFO (OverseerStateUpdate-73275888448634898-127.0.0.1:32996_solr-n_0000000000) [n:127.0.0.1:32996_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:56776/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 674319 INFO (OverseerStateUpdate-73275888448634898-127.0.0.1:32996_solr-n_0000000000) [n:127.0.0.1:32996_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:48615/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 674320 INFO (OverseerStateUpdate-73275888448634898-127.0.0.1:32996_solr-n_0000000000) [n:127.0.0.1:32996_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:38116/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 674322 INFO (OverseerStateUpdate-73275888448634898-127.0.0.1:32996_solr-n_0000000000) [n:127.0.0.1:32996_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36661/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 674324 INFO (OverseerStateUpdate-73275888448634898-127.0.0.1:32996_solr-n_0000000000) [n:127.0.0.1:32996_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard3_replica_n8",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33961/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 674325 INFO (OverseerStateUpdate-73275888448634898-127.0.0.1:32996_solr-n_0000000000) [n:127.0.0.1:32996_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard3",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard3_replica_n10",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:32996/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 674327 INFO (OverseerStateUpdate-73275888448634898-127.0.0.1:32996_solr-n_0000000000) [n:127.0.0.1:32996_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard4_replica_n12",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:32996/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 674329 INFO (OverseerStateUpdate-73275888448634898-127.0.0.1:32996_solr-n_0000000000) [n:127.0.0.1:32996_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard4",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard4_replica_n14",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33961/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 674331 INFO (OverseerStateUpdate-73275888448634898-127.0.0.1:32996_solr-n_0000000000) [n:127.0.0.1:32996_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard5_replica_n16",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36661/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 674333 INFO (OverseerStateUpdate-73275888448634898-127.0.0.1:32996_solr-n_0000000000) [n:127.0.0.1:32996_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"replacenodetest_coll_notarget",
[junit4] 2> "shard":"shard5",
[junit4] 2> "core":"replacenodetest_coll_notarget_shard5_replica_n18",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:38116/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 674543 INFO (qtp342457549-6499) [n:127.0.0.1:38116_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard2_replica_n4&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 674543 INFO (qtp152333974-6511) [n:127.0.0.1:36661_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node9&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard2_replica_n6&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 674543 INFO (qtp727093809-6516) [n:127.0.0.1:56776_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard1_replica_n1&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 674543 INFO (qtp342457549-6499) [n:127.0.0.1:38116_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 674544 INFO (qtp152333974-6511) [n:127.0.0.1:36661_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 674544 INFO (qtp1266990277-6495) [n:127.0.0.1:48615_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard1_replica_n2&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 674544 INFO (qtp1266990277-6495) [n:127.0.0.1:48615_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 674544 INFO (qtp727093809-6516) [n:127.0.0.1:56776_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 674544 INFO (qtp1976465646-6498) [n:127.0.0.1:33961_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node11&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard3_replica_n8&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 674545 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node13&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard3_replica_n10&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 674545 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node15&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard4_replica_n12&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 674545 INFO (qtp1976465646-6498) [n:127.0.0.1:33961_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 674545 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 674546 INFO (qtp1976465646-6496) [n:127.0.0.1:33961_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node17&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard4_replica_n14&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard4&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 674546 INFO (qtp342457549-6503) [n:127.0.0.1:38116_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node20&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard5_replica_n18&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard5&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 674547 INFO (qtp152333974-6508) [n:127.0.0.1:36661_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node19&collection.configName=conf1&newCollection=true&name=replacenodetest_coll_notarget_shard5_replica_n16&action=CREATE&numShards=5&collection=replacenodetest_coll_notarget&shard=shard5&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 674657 INFO (zkCallback-2810-thread-2-processing-n:127.0.0.1:38116_solr) [n:127.0.0.1:38116_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 674657 INFO (zkCallback-2808-thread-1-processing-n:127.0.0.1:56776_solr) [n:127.0.0.1:56776_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 674657 INFO (zkCallback-2802-thread-1-processing-n:127.0.0.1:36661_solr) [n:127.0.0.1:36661_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 674657 INFO (zkCallback-2806-thread-1-processing-n:127.0.0.1:33961_solr) [n:127.0.0.1:33961_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 674658 INFO (zkCallback-2808-thread-2-processing-n:127.0.0.1:56776_solr) [n:127.0.0.1:56776_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 674657 INFO (zkCallback-2804-thread-1-processing-n:127.0.0.1:48615_solr) [n:127.0.0.1:48615_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 674658 INFO (zkCallback-2812-thread-1-processing-n:127.0.0.1:32996_solr) [n:127.0.0.1:32996_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 674661 INFO (zkCallback-2802-thread-1-processing-n:127.0.0.1:36661_solr) [n:127.0.0.1:36661_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 674661 INFO (zkCallback-2802-thread-2-processing-n:127.0.0.1:36661_solr) [n:127.0.0.1:36661_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 674661 INFO (zkCallback-2806-thread-2-processing-n:127.0.0.1:33961_solr) [n:127.0.0.1:33961_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 674663 INFO (zkCallback-2812-thread-2-processing-n:127.0.0.1:32996_solr) [n:127.0.0.1:32996_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 674663 INFO (zkCallback-2804-thread-2-processing-n:127.0.0.1:48615_solr) [n:127.0.0.1:48615_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/replacenodetest_coll_notarget/state.json] for collection [replacenodetest_coll_notarget] has occurred - updating... (live nodes size: [6])
[junit4] 2> 675572 INFO (qtp727093809-6516) [n:127.0.0.1:56776_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 675573 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 675573 INFO (qtp1976465646-6496) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 675578 INFO (qtp152333974-6508) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 675582 INFO (qtp1266990277-6495) [n:127.0.0.1:48615_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 675598 INFO (qtp1976465646-6498) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 675599 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 675600 INFO (qtp152333974-6508) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard5_replica_n16] Schema name=cloud-dynamic
[junit4] 2> 675601 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard4_replica_n12] Schema name=cloud-dynamic
[junit4] 2> 675604 INFO (qtp342457549-6499) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 675605 INFO (qtp152333974-6511) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 675609 INFO (qtp342457549-6503) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 675621 INFO (qtp727093809-6516) [n:127.0.0.1:56776_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard1_replica_n1] Schema name=cloud-dynamic
[junit4] 2> 675626 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard3_replica_n10] Schema name=cloud-dynamic
[junit4] 2> 675628 INFO (qtp1266990277-6495) [n:127.0.0.1:48615_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard1_replica_n2] Schema name=cloud-dynamic
[junit4] 2> 675634 INFO (qtp342457549-6499) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard2_replica_n4] Schema name=cloud-dynamic
[junit4] 2> 675635 INFO (qtp342457549-6503) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard5_replica_n18] Schema name=cloud-dynamic
[junit4] 2> 675661 INFO (qtp1976465646-6496) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard4_replica_n14] Schema name=cloud-dynamic
[junit4] 2> 675662 INFO (qtp152333974-6511) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard2_replica_n6] Schema name=cloud-dynamic
[junit4] 2> 675663 INFO (qtp1976465646-6498) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.s.IndexSchema [replacenodetest_coll_notarget_shard3_replica_n8] Schema name=cloud-dynamic
[junit4] 2> 675693 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 675693 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard3_replica_n10' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 675694 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32996.solr.core.replacenodetest_coll_notarget.shard3.replica_n10' (registry 'solr.core.replacenodetest_coll_notarget.shard3.replica_n10') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 675694 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 675694 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard3_replica_n10] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node2/replacenodetest_coll_notarget_shard3_replica_n10], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node2/./replacenodetest_coll_notarget_shard3_replica_n10/data/]
[junit4] 2> 675696 INFO (qtp152333974-6508) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 675696 INFO (qtp152333974-6508) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard5_replica_n16' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 675696 INFO (qtp152333974-6508) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36661.solr.core.replacenodetest_coll_notarget.shard5.replica_n16' (registry 'solr.core.replacenodetest_coll_notarget.shard5.replica_n16') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 675696 INFO (qtp152333974-6508) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 675696 INFO (qtp152333974-6508) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard5 r:core_node19 x:replacenodetest_coll_notarget_shard5_replica_n16] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard5_replica_n16] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node3/replacenodetest_coll_notarget_shard5_replica_n16], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node3/./replacenodetest_coll_notarget_shard5_replica_n16/data/]
[junit4] 2> 675704 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 675704 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard4_replica_n12' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 675707 INFO (qtp342457549-6503) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 675707 INFO (qtp342457549-6503) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard5_replica_n18' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 675707 INFO (qtp342457549-6503) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38116.solr.core.replacenodetest_coll_notarget.shard5.replica_n18' (registry 'solr.core.replacenodetest_coll_notarget.shard5.replica_n18') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 675707 INFO (qtp342457549-6503) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 675707 INFO (qtp342457549-6503) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard5 r:core_node20 x:replacenodetest_coll_notarget_shard5_replica_n18] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard5_replica_n18] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node5/replacenodetest_coll_notarget_shard5_replica_n18], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node5/./replacenodetest_coll_notarget_shard5_replica_n18/data/]
[junit4] 2> 675723 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32996.solr.core.replacenodetest_coll_notarget.shard4.replica_n12' (registry 'solr.core.replacenodetest_coll_notarget.shard4.replica_n12') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 675723 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 675723 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard4_replica_n12] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node2/replacenodetest_coll_notarget_shard4_replica_n12], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node2/./replacenodetest_coll_notarget_shard4_replica_n12/data/]
[junit4] 2> 675742 INFO (qtp1266990277-6495) [n:127.0.0.1:48615_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 675742 INFO (qtp1266990277-6495) [n:127.0.0.1:48615_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard1_replica_n2' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 675743 INFO (qtp152333974-6511) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 675743 INFO (qtp152333974-6511) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard2_replica_n6' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 675743 INFO (qtp152333974-6511) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36661.solr.core.replacenodetest_coll_notarget.shard2.replica_n6' (registry 'solr.core.replacenodetest_coll_notarget.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 675743 INFO (qtp152333974-6511) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 675743 INFO (qtp152333974-6511) [n:127.0.0.1:36661_solr c:replacenodetest_coll_notarget s:shard2 r:core_node9 x:replacenodetest_coll_notarget_shard2_replica_n6] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard2_replica_n6] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node3/replacenodetest_coll_notarget_shard2_replica_n6], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node3/./replacenodetest_coll_notarget_shard2_replica_n6/data/]
[junit4] 2> 675745 INFO (qtp342457549-6499) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 675745 INFO (qtp342457549-6499) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard2_replica_n4' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 675745 INFO (qtp342457549-6499) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38116.solr.core.replacenodetest_coll_notarget.shard2.replica_n4' (registry 'solr.core.replacenodetest_coll_notarget.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 675746 INFO (qtp1976465646-6496) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 675747 INFO (qtp1976465646-6496) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard4_replica_n14' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 675747 INFO (qtp1976465646-6496) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33961.solr.core.replacenodetest_coll_notarget.shard4.replica_n14' (registry 'solr.core.replacenodetest_coll_notarget.shard4.replica_n14') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 675747 INFO (qtp1976465646-6496) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 675747 INFO (qtp1976465646-6496) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard4 r:core_node17 x:replacenodetest_coll_notarget_shard4_replica_n14] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard4_replica_n14] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node4/replacenodetest_coll_notarget_shard4_replica_n14], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node4/./replacenodetest_coll_notarget_shard4_replica_n14/data/]
[junit4] 2> 675749 INFO (qtp342457549-6499) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 675749 INFO (qtp342457549-6499) [n:127.0.0.1:38116_solr c:replacenodetest_coll_notarget s:shard2 r:core_node7 x:replacenodetest_coll_notarget_shard2_replica_n4] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard2_replica_n4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node5/replacenodetest_coll_notarget_shard2_replica_n4], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node5/./replacenodetest_coll_notarget_shard2_replica_n4/data/]
[junit4] 2> 675752 INFO (qtp1266990277-6495) [n:127.0.0.1:48615_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_48615.solr.core.replacenodetest_coll_notarget.shard1.replica_n2' (registry 'solr.core.replacenodetest_coll_notarget.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 675752 INFO (qtp1266990277-6495) [n:127.0.0.1:48615_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 675752 INFO (qtp1266990277-6495) [n:127.0.0.1:48615_solr c:replacenodetest_coll_notarget s:shard1 r:core_node5 x:replacenodetest_coll_notarget_shard1_replica_n2] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard1_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node1/replacenodetest_coll_notarget_shard1_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node1/./replacenodetest_coll_notarget_shard1_replica_n2/data/]
[junit4] 2> 675765 INFO (qtp1976465646-6498) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 675765 INFO (qtp1976465646-6498) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard3_replica_n8' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 675765 INFO (qtp1976465646-6498) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33961.solr.core.replacenodetest_coll_notarget.shard3.replica_n8' (registry 'solr.core.replacenodetest_coll_notarget.shard3.replica_n8') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 675765 INFO (qtp1976465646-6498) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 675765 INFO (qtp1976465646-6498) [n:127.0.0.1:33961_solr c:replacenodetest_coll_notarget s:shard3 r:core_node11 x:replacenodetest_coll_notarget_shard3_replica_n8] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard3_replica_n8] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node4/replacenodetest_coll_notarget_shard3_replica_n8], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node4/./replacenodetest_coll_notarget_shard3_replica_n8/data/]
[junit4] 2> 675783 INFO (qtp727093809-6516) [n:127.0.0.1:56776_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 675783 INFO (qtp727093809-6516) [n:127.0.0.1:56776_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'replacenodetest_coll_notarget_shard1_replica_n1' using configuration from collection replacenodetest_coll_notarget, trusted=true
[junit4] 2> 675783 INFO (qtp727093809-6516) [n:127.0.0.1:56776_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_56776.solr.core.replacenodetest_coll_notarget.shard1.replica_n1' (registry 'solr.core.replacenodetest_coll_notarget.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7ea44092
[junit4] 2> 675784 INFO (qtp727093809-6516) [n:127.0.0.1:56776_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 675784 INFO (qtp727093809-6516) [n:127.0.0.1:56776_solr c:replacenodetest_coll_notarget s:shard1 r:core_node3 x:replacenodetest_coll_notarget_shard1_replica_n1] o.a.s.c.SolrCore [[replacenodetest_coll_notarget_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node6/replacenodetest_coll_notarget_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.ReplaceNodeNoTargetTest_76F376A29D9976EF-001/tempDir-001/node6/./replacenodetest_coll_notarget_shard1_replica_n1/data/]
[junit4] 2> 675891 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 675891 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 675915 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 675915 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 675933 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 675933 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 675942 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 675942 INFO (qtp1329496417-6494) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard3 r:core_node13 x:replacenodetest_coll_notarget_shard3_replica_n10] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 675943 INFO (qtp1329496417-6480) [n:127.0.0.1:32996_solr c:replacenodetest_coll_notarget s:shard4 r:core_node15 x:replacenodetest_coll_notarget_shard4_replica_n12] o.a.s.s.SolrIndexSearcher Opening [Searcher@7cb34d12[replacenodetest_coll_notarget_shard4_replica_n12] main]
[junit
[...truncated too long message...]
28))), a_t=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), id=PostingsFormat(name=Memory), 5grand3_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 1parent_f1_s=PostingsFormat(name=Memory), 13parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 23parent_f1_s=PostingsFormat(name=Memory), _root_=PostingsFormat(name=Memory), 5child2_s=PostingsFormat(name=Memory), 29parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 3child0_s=PostingsFormat(name=Memory), 4child3_s=PostingsFormat(name=Memory), 12parent_f1_s=PostingsFormat(name=Memory), 22parent_f1_s=PostingsFormat(name=Memory), 1grand0_s=PostingsFormat(name=LuceneVarGapFixedInterval), 7parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 2child1_s=PostingsFormat(name=Memory), 3parent_f2_s=PostingsFormat(name=Memory), 6grand3_s=PostingsFormat(name=LuceneVarGapFixedInterval), 5grand2_s=PostingsFormat(name=Memory), 11parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 21parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 4grand1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 7grand4_s=PostingsFormat(name=Memory), 3grand0_s=PostingsFormat(name=Memory), 9grand6_s=PostingsFormat(name=Memory), 8grand1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 9grand2_s=PostingsFormat(name=Memory), 8grand5_s=PostingsFormat(name=LuceneVarGapFixedInterval), 9parent_f1_s=PostingsFormat(name=Memory), 14parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 20parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 1parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 7grand0_s=PostingsFormat(name=Memory), text_t=PostingsFormat(name=Memory), 24parent_f2_s=PostingsFormat(name=Memory), 13parent_f2_s=PostingsFormat(name=Memory), type_s=PostingsFormat(name=Memory), 23parent_f2_s=PostingsFormat(name=Memory), 5child1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 10parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 22parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 4child0_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 12parent_f2_s=PostingsFormat(name=Memory), 5grand1_s=PostingsFormat(name=Memory), 6grand4_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 11parent_f1_s=PostingsFormat(name=Memory), 21parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 4grand2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 7grand3_s=PostingsFormat(name=Memory), 8grand2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 9grand5_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 8grand6_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 9grand1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 14parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 20parent_f2_s=PostingsFormat(name=Memory), 24parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 5parent_f1_s=PostingsFormat(name=Memory), 6grand0_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 10parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 5child0_s=PostingsFormat(name=LuceneVarGapFixedInterval), 8parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 27parent_f2_s=PostingsFormat(name=Memory), 2parent_f2_s=PostingsFormat(name=Memory), 4child1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 2grand1_s=PostingsFormat(name=Memory), 17parent_f2_s=PostingsFormat(name=Memory), 3child2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 16parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 26parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 1child0_s=PostingsFormat(name=LuceneVarGapFixedInterval), 7grand6_s=PostingsFormat(name=LuceneVarGapFixedInterval), 18parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 28parent_f1_s=PostingsFormat(name=Memory), 5grand0_s=PostingsFormat(name=LuceneVarGapFixedInterval), 6grand5_s=PostingsFormat(name=Memory), 7grand2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 15parent_f1_s=PostingsFormat(name=Memory), 4grand3_s=PostingsFormat(name=Memory), 9grand8_s=PostingsFormat(name=LuceneVarGapFixedInterval), 8grand3_s=PostingsFormat(name=Memory), 6parent_f1_s=PostingsFormat(name=Memory), 25parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval), 9grand4_s=PostingsFormat(name=LuceneVarGapFixedInterval), 8grand7_s=PostingsFormat(name=Memory), 0parent_f1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 3grand2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 9grand0_s=PostingsFormat(name=LuceneVarGapFixedInterval), 19parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 4parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 5child4_s=PostingsFormat(name=LuceneVarGapFixedInterval), 5grand4_s=PostingsFormat(name=LuceneVarGapFixedInterval), 6grand1_s=PostingsFormat(name=Memory), 5parent_f2_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 16parent_f1_s=PostingsFormat(name=Memory), 8parent_f2_s=PostingsFormat(name=LuceneVarGapFixedInterval), 29parent_f2_s=PostingsFormat(name=Memory), 2parent_f1_s=PostingsFormat(name=Memory), 4child2_s=PostingsFormat(name=Memory), rnd_b=PostingsFormat(name=Memory), 27parent_f1_s=PostingsFormat(name=Memory), 17parent_f1_s=PostingsFormat(name=Memory), 2grand0_s=PostingsFormat(name=Memory), 3child1_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 7parent_f2_s=PostingsFormat(name=Memory), 26parent_f1_s=PostingsFormat(name=Memory), 2child0_s=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), 3parent_f1_s=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{_version_=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=975, maxMBSortInHeap=7.185705474415596, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@4a54bd5e), locale=fr-FR, timezone=America/Boise
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=229770944,total=521666560
[junit4] 2> NOTE: All tests run in this JVM: [TestSearcherReuse, BasicZkTest, TestFunctionQuery, SOLR749Test, ManagedSchemaRoundRobinCloudTest, TestBinaryResponseWriter, HdfsBasicDistributedZk2Test, TestComplexPhraseQParserPlugin, HdfsWriteToMultipleCollectionsTest, TestSimpleTrackingShardHandler, SolrSlf4jReporterTest, TestExactSharedStatsCache, WrapperMergePolicyFactoryTest, AlternateDirectoryTest, TestRTGBase, BlockDirectoryTest, TestDFISimilarityFactory, ClassificationUpdateProcessorTest, BlockJoinFacetDistribTest, TestPostingsSolrHighlighter, TestPartialUpdateDeduplication, TestSolrCloudWithHadoopAuthPlugin, QueryParsingTest, SimplePostToolTest, TestManagedSchemaThreadSafety, TestOrdValues, DeleteNodeTest, LoggingHandlerTest, TestLRUCache, CdcrRequestHandlerTest, TestBM25SimilarityFactory, TestHalfAndHalfDocValues, TestFastLRUCache, DistribJoinFromCollectionTest, CollectionsAPIDistributedZkTest, SuggestComponentContextFilterQueryTest, HttpTriggerListenerTest, TestSearchPerf, DeleteStatusTest, SSLMigrationTest, OverseerModifyCollectionTest, TestReplicationHandler, HdfsLockFactoryTest, DirectSolrConnectionTest, TestMissingGroups, DistributedFacetPivotWhiteBoxTest, ConfigSetsAPITest, TestTolerantUpdateProcessorCloud, ZkFailoverTest, SyncSliceTest, TestElisionMultitermQuery, UpdateRequestProcessorFactoryTest, TestSolrXml, DistanceUnitsTest, SecurityConfHandlerTest, TestPolicyCloud, TestInitParams, TestEmbeddedSolrServerAdminHandler, TestUseDocValuesAsStored2, FileUtilsTest, TestSweetSpotSimilarityFactory, CoreAdminCreateDiscoverTest, TestSolrIndexConfig, TestAddFieldRealTimeGet, ChaosMonkeySafeLeaderTest, VersionInfoTest, TestCoreAdminApis, TestCryptoKeys, TestConfigSetsAPI, AtomicUpdatesTest, PathHierarchyTokenizerFactoryTest, MetricsConfigTest, TestDownShardTolerantSearch, HdfsThreadLeakTest, HttpPartitionOnCommitTest, TestAnalyzeInfixSuggestions, SystemInfoHandlerTest, SignatureUpdateProcessorFactoryTest, DocumentBuilderTest, TestOverriddenPrefixQueryForCustomFieldType, TestXmlQParser, TestInPlaceUpdatesDistrib, TestFoldingMultitermQuery, TestSchemaResource, TestQueryUtils, DateRangeFieldTest, ZkStateReaderTest, TestConfigOverlay, TestApiFramework, TestSSLRandomization, TestQueryWrapperFilter, TestTestInjection, SoftAutoCommitTest, OverseerTest, CreateRoutedAliasTest, PrimitiveFieldTypeTest, AutoAddReplicasIntegrationTest, PeerSyncTest, RegexBytesRefFilterTest, DistributedMLTComponentTest, SolrCoreTest, LeaderInitiatedRecoveryOnShardRestartTest, TestScoreJoinQPNoScore, HdfsDirectoryFactoryTest, TestRandomRequestDistribution, TestTrie, SuggesterTest, TestSolr4Spatial2, TestInitQParser, SolrMetricReporterTest, ExplicitHLLTest, ReplicationFactorTest, SolrGraphiteReporterTest, TestJettySolrRunner, TestCloudJSONFacetJoinDomain, TestSolr4Spatial, TestDynamicFieldResource, TestStandardQParsers, TestGenericDistributedQueue, TestIntervalFaceting, TestSolrConfigHandlerCloud, TestRecovery, HLLUtilTest, MergeStrategyTest, TestReRankQParserPlugin, BasicAuthIntegrationTest, TestCloudManagedSchema, UnloadDistributedZkTest, OpenCloseCoreStressTest, ShardRoutingTest, FullSolrCloudDistribCmdsTest]
[junit4] Completed [300/776 (2!)] on J1 in 62.58s, 1 test, 1 failure <<< FAILURES!
[...truncated 49829 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2308 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2308/
2 tests failed.
FAILED: org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Error from server at https://127.0.0.1:42540/solr/awhollynewcollection_0_shard1_replica_n1: ClusterState says we are the leader (https://127.0.0.1:42540/solr/awhollynewcollection_0_shard1_replica_n1), but locally we don't think so. Request came from null
Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at https://127.0.0.1:42540/solr/awhollynewcollection_0_shard1_replica_n1: ClusterState says we are the leader (https://127.0.0.1:42540/solr/awhollynewcollection_0_shard1_replica_n1), but locally we don't think so. Request came from null
at __randomizedtesting.SeedInfo.seed([8F865384DA767929:C7F32730DC4556BC]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:550)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1013)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:884)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:946)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:946)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:946)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:946)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:946)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:463)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:42540/solr/awhollynewcollection_0_shard1_replica_n1: ClusterState says we are the leader (https://127.0.0.1:42540/solr/awhollynewcollection_0_shard1_replica_n1), but locally we don't think so. Request came from null
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:527)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventFromRestoredState
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([8F865384DA767929:8FB0E744F37BDD43]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventFromRestoredState(TriggerIntegrationTest.java:760)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12665 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_8F865384DA767929-001/init-core-data-001
[junit4] 2> 1200827 WARN (SUITE-TriggerIntegrationTest-seed#[8F865384DA767929]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 1200827 INFO (SUITE-TriggerIntegrationTest-seed#[8F865384DA767929]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1200841 INFO (SUITE-TriggerIntegrationTest-seed#[8F865384DA767929]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1200842 INFO (SUITE-TriggerIntegrationTest-seed#[8F865384DA767929]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 1200842 INFO (SUITE-TriggerIntegrationTest-seed#[8F865384DA767929]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_8F865384DA767929-001/tempDir-001
[junit4] 2> 1200842 INFO (SUITE-TriggerIntegrationTest-seed#[8F865384DA767929]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1200853 INFO (Thread-2372) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1200853 INFO (Thread-2372) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1200855 ERROR (Thread-2372) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1200953 INFO (SUITE-TriggerIntegrationTest-seed#[8F865384DA767929]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:44676
[junit4] 2> 1200998 INFO (zkConnectionManagerCallback-875-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1201024 INFO (jetty-launcher-872-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1201026 INFO (jetty-launcher-872-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1201026 INFO (jetty-launcher-872-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1201026 INFO (jetty-launcher-872-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1201026 INFO (jetty-launcher-872-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1201027 INFO (jetty-launcher-872-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@72f1060{/solr,null,AVAILABLE}
[junit4] 2> 1201105 INFO (jetty-launcher-872-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1201105 INFO (jetty-launcher-872-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1201105 INFO (jetty-launcher-872-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1201105 INFO (jetty-launcher-872-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2b268fc8{/solr,null,AVAILABLE}
[junit4] 2> 1201106 INFO (jetty-launcher-872-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@75208f92{SSL,[ssl, http/1.1]}{127.0.0.1:37311}
[junit4] 2> 1201106 INFO (jetty-launcher-872-thread-2) [ ] o.e.j.s.Server Started @1211430ms
[junit4] 2> 1201106 INFO (jetty-launcher-872-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37311}
[junit4] 2> 1201106 ERROR (jetty-launcher-872-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1201106 INFO (jetty-launcher-872-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1201106 INFO (jetty-launcher-872-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1201106 INFO (jetty-launcher-872-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1201106 INFO (jetty-launcher-872-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T02:44:14.703Z
[junit4] 2> 1201137 INFO (jetty-launcher-872-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@29a564f6{SSL,[ssl, http/1.1]}{127.0.0.1:38684}
[junit4] 2> 1201137 INFO (jetty-launcher-872-thread-1) [ ] o.e.j.s.Server Started @1211461ms
[junit4] 2> 1201137 INFO (jetty-launcher-872-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38684}
[junit4] 2> 1201137 ERROR (jetty-launcher-872-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1201138 INFO (jetty-launcher-872-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1201138 INFO (jetty-launcher-872-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1201138 INFO (jetty-launcher-872-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1201138 INFO (jetty-launcher-872-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T02:44:14.735Z
[junit4] 2> 1201157 INFO (zkConnectionManagerCallback-877-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1201162 INFO (jetty-launcher-872-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1201194 INFO (zkConnectionManagerCallback-879-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1201198 INFO (jetty-launcher-872-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1201239 INFO (jetty-launcher-872-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44676/solr
[junit4] 2> 1201263 INFO (zkConnectionManagerCallback-883-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1201284 INFO (jetty-launcher-872-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44676/solr
[junit4] 2> 1201359 INFO (zkConnectionManagerCallback-887-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1201360 INFO (zkConnectionManagerCallback-889-thread-1-processing-n:127.0.0.1:37311_solr) [n:127.0.0.1:37311_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1201389 INFO (zkConnectionManagerCallback-891-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1201941 INFO (jetty-launcher-872-thread-2) [n:127.0.0.1:37311_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1201941 INFO (jetty-launcher-872-thread-2) [n:127.0.0.1:37311_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37311_solr
[junit4] 2> 1201975 INFO (jetty-launcher-872-thread-2) [n:127.0.0.1:37311_solr ] o.a.s.c.Overseer Overseer (id=72100484426235909-127.0.0.1:37311_solr-n_0000000000) starting
[junit4] 2> 1202002 INFO (jetty-launcher-872-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1202003 INFO (jetty-launcher-872-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38684_solr
[junit4] 2> 1202089 INFO (zkCallback-888-thread-1-processing-n:127.0.0.1:37311_solr) [n:127.0.0.1:37311_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1202089 INFO (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1202276 INFO (jetty-launcher-872-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38684.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1202369 INFO (jetty-launcher-872-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38684.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1202369 INFO (jetty-launcher-872-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38684.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1202371 INFO (jetty-launcher-872-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_8F865384DA767929-001/tempDir-001/node1/.
[junit4] 2> 1202820 DEBUG (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1202829 INFO (jetty-launcher-872-thread-2) [n:127.0.0.1:37311_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37311_solr
[junit4] 2> 1202829 DEBUG (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1202830 DEBUG (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:38684_solr]
[junit4] 2> 1202830 DEBUG (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1202830 DEBUG (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1202865 INFO (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1202865 INFO (zkCallback-888-thread-1-processing-n:127.0.0.1:37311_solr) [n:127.0.0.1:37311_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1202867 DEBUG (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1202867 DEBUG (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1202868 DEBUG (ScheduledTrigger-1702-thread-1) [n:127.0.0.1:37311_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1202998 INFO (jetty-launcher-872-thread-2) [n:127.0.0.1:37311_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37311.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1203034 INFO (jetty-launcher-872-thread-2) [n:127.0.0.1:37311_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37311.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1203034 INFO (jetty-launcher-872-thread-2) [n:127.0.0.1:37311_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37311.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1203035 INFO (jetty-launcher-872-thread-2) [n:127.0.0.1:37311_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_8F865384DA767929-001/tempDir-001/node2/.
[junit4] 2> 1203207 INFO (zkConnectionManagerCallback-897-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1203223 INFO (zkConnectionManagerCallback-901-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1203224 INFO (SUITE-TriggerIntegrationTest-seed#[8F865384DA767929]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1203225 INFO (SUITE-TriggerIntegrationTest-seed#[8F865384DA767929]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:44676/solr ready
[junit4] 2> 1203309 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeMarkersRegistration
[junit4] 2> 1203310 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 1203311 DEBUG (zkCallback-888-thread-1-processing-n:127.0.0.1:37311_solr) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1203311 DEBUG (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1203314 DEBUG (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1203314 DEBUG (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1203314 DEBUG (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1203420 INFO (qtp1133844329-5708) [n:127.0.0.1:37311_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1203498 INFO (qtp1133844329-5708) [n:127.0.0.1:37311_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=78
[junit4] 2> 1203501 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1203516 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1203516 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1203516 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1203517 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5608f119{/solr,null,AVAILABLE}
[junit4] 2> 1203533 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6eb44f6d{SSL,[ssl, http/1.1]}{127.0.0.1:44763}
[junit4] 2> 1203533 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.Server Started @1213857ms
[junit4] 2> 1203533 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44763}
[junit4] 2> 1203533 ERROR (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1203533 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1203533 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1203533 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1203533 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T02:44:17.130Z
[junit4] 2> 1203609 INFO (zkConnectionManagerCallback-903-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1203630 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1203899 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44676/solr
[junit4] 2> 1203914 INFO (zkConnectionManagerCallback-907-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1204032 INFO (zkConnectionManagerCallback-909-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1204065 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:44763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1204067 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:44763_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1204068 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:44763_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44763_solr
[junit4] 2> 1204106 INFO (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1204178 INFO (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1204244 INFO (zkCallback-900-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1204260 INFO (zkCallback-888-thread-2-processing-n:127.0.0.1:37311_solr) [n:127.0.0.1:37311_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1204447 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:44763_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44763.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1204500 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:44763_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44763.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1204501 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:44763_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44763.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1204502 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:44763_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_8F865384DA767929-001/tempDir-001/node3/.
[junit4] 2> 1204809 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.a.TriggerIntegrationTest ====== KILL OVERSEER 1
[junit4] 2> 1204842 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@75208f92{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1204970 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=731056919
[junit4] 2> 1204970 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1204984 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@741ff32a: rootName = solr_37311, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@67a5836f
[junit4] 2> 1205039 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1205039 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@17260a72: rootName = solr_37311, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2fd787d9
[junit4] 2> 1205096 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1205096 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@19d731ea: rootName = solr_37311, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@242430ac
[junit4] 2> 1205108 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37311_solr
[junit4] 2> 1205109 INFO (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1205109 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1205122 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1205122 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.Overseer Overseer (id=72100484426235909-127.0.0.1:37311_solr-n_0000000000) closing
[junit4] 2> 1205122 INFO (OverseerStateUpdate-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37311_solr
[junit4] 2> 1205143 WARN (OverseerAutoScalingTriggerThread-72100484426235909-127.0.0.1:37311_solr-n_0000000000) [n:127.0.0.1:37311_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1205144 DEBUG (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1205143 INFO (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1205143 INFO (zkCallback-900-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1205143 INFO (zkCallback-888-thread-1-processing-n:127.0.0.1:37311_solr) [n:127.0.0.1:37311_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1205145 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2b268fc8{/solr,null,UNAVAILABLE}
[junit4] 2> 1205145 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1205207 INFO (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38684_solr
[junit4] 2> 1205333 INFO (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.Overseer Overseer (id=72100484426235910-127.0.0.1:38684_solr-n_0000000001) starting
[junit4] 2> 1205592 DEBUG (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1205592 DEBUG (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1205593 DEBUG (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:38684_solr, 127.0.0.1:44763_solr]
[junit4] 2> 1205593 DEBUG (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion -1
[junit4] 2> 1205593 DEBUG (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1205594 DEBUG (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1205594 DEBUG (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1205595 DEBUG (ScheduledTrigger-1715-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1206602 DEBUG (ScheduledTrigger-1715-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1207607 DEBUG (ScheduledTrigger-1715-thread-2) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1208623 DEBUG (ScheduledTrigger-1715-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1209634 DEBUG (ScheduledTrigger-1715-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1210199 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.a.TriggerIntegrationTest ====== ADD TRIGGERS
[junit4] 2> 1210457 INFO (qtp1350247202-5694) [n:127.0.0.1:38684_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1210651 DEBUG (ScheduledTrigger-1715-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1210733 INFO (qtp1141431223-5764) [n:127.0.0.1:44763_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1210734 DEBUG (qtp1350247202-5699) [n:127.0.0.1:38684_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1210768 INFO (qtp1350247202-5699) [n:127.0.0.1:38684_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=349
[junit4] 2> 1210805 DEBUG (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1210821 DEBUG (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:38684_solr, 127.0.0.1:44763_solr]
[junit4] 2> 1210821 DEBUG (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerMR instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestEventMarkerAction}], enabled=true}
[junit4] 2> 1210821 DEBUG (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:38684_solr, 127.0.0.1:44763_solr]
[junit4] 2> 1210851 INFO (qtp1350247202-5700) [n:127.0.0.1:38684_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1210853 DEBUG (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1210855 INFO (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventMarkerAction init
[junit4] 2> 1210891 DEBUG (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1210903 INFO (qtp1141431223-5763) [n:127.0.0.1:44763_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1210903 DEBUG (qtp1350247202-5697) [n:127.0.0.1:38684_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1210906 INFO (qtp1350247202-5697) [n:127.0.0.1:38684_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=137
[junit4] 2> 1210919 DEBUG (ScheduledTrigger-1715-thread-3) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerMR
[junit4] 2> 1210936 DEBUG (ScheduledTrigger-1715-thread-3) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1210942 DEBUG (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1210942 DEBUG (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:38684_solr, 127.0.0.1:44763_solr]
[junit4] 2> 1210942 DEBUG (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:38684_solr, 127.0.0.1:44763_solr]
[junit4] 2> 1210942 DEBUG (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerMR instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestEventMarkerAction}], enabled=true}
[junit4] 2> 1210942 DEBUG (zkCallback-890-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:38684_solr, 127.0.0.1:44763_solr]
[junit4] 2> 1210947 DEBUG (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1210948 INFO (qtp1350247202-5699) [n:127.0.0.1:38684_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1211016 INFO (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventMarkerAction init
[junit4] 2> 1211018 DEBUG (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1211018 DEBUG (ScheduledTrigger-1715-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_triggerMR with currently live nodes: 2
[junit4] 2> 1211148 INFO (qtp1350247202-5699) [n:127.0.0.1:38684_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=200
[junit4] 2> 1211166 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.a.TriggerIntegrationTest ====== ADD NODE 1
[junit4] 2> 1211168 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1211292 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1211292 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1211292 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1211293 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@27485f1c{/solr,null,AVAILABLE}
[junit4] 2> 1211293 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@59d97321{SSL,[ssl, http/1.1]}{127.0.0.1:43653}
[junit4] 2> 1211293 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.Server Started @1221618ms
[junit4] 2> 1211293 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43653}
[junit4] 2> 1211294 ERROR (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1211294 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1211294 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1211294 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1211294 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T02:44:24.891Z
[junit4] 2> 1211333 INFO (zkConnectionManagerCallback-913-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1211345 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1211463 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44676/solr
[junit4] 2> 1211512 INFO (zkConnectionManagerCallback-917-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1211572 INFO (zkConnectionManagerCallback-919-thread-1-processing-n:127.0.0.1:43653_solr) [n:127.0.0.1:43653_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1211654 DEBUG (ScheduledTrigger-1715-thread-2) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1211693 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:43653_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1211707 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:43653_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1211709 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:43653_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43653_solr
[junit4] 2> 1211727 INFO (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1211731 INFO (zkCallback-900-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1211767 INFO (zkCallback-890-thread-2-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1211796 INFO (zkCallback-918-thread-1-processing-n:127.0.0.1:43653_solr) [n:127.0.0.1:43653_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1212096 DEBUG (ScheduledTrigger-1715-thread-4) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerMR
[junit4] 2> 1212097 DEBUG (ScheduledTrigger-1715-thread-4) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1212097 DEBUG (ScheduledTrigger-1715-thread-4) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:43653_solr at time 654466631211934
[junit4] 2> 1212198 DEBUG (ScheduledTrigger-1715-thread-3) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_triggerMR with currently live nodes: 3
[junit4] 2> 1212251 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:43653_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43653.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1212289 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:43653_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43653.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1212289 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:43653_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43653.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1212291 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [n:127.0.0.1:43653_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_8F865384DA767929-001/tempDir-001/node4/.
[junit4] 2> 1212654 DEBUG (ScheduledTrigger-1715-thread-1) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1213023 INFO (OverseerCollectionConfigSetProcessor-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1213206 DEBUG (ScheduledTrigger-1715-thread-2) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerMR
[junit4] 2> 1213206 DEBUG (ScheduledTrigger-1715-thread-2) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1213206 DEBUG (ScheduledTrigger-1715-thread-2) [n:127.0.0.1:38684_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerMR firing registered processor for nodes: [127.0.0.1:43653_solr] added at times [654466631211934], now=654467740940761
[junit4] 2> 1213207 DEBUG (ScheduledTrigger-1715-thread-2) [n:127.0.0.1:38684_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"2533be3121b9eT3mumm590e6fkqoi6wywb8tlbw",
[junit4] 2> "source":"node_added_triggerMR",
[junit4] 2> "eventTime":654466631211934,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[654466631211934],
[junit4] 2> "nodeNames":["127.0.0.1:43653_solr"]}}
[junit4] 2> 1213207 DEBUG (ScheduledTrigger-1715-thread-2) [n:127.0.0.1:38684_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_triggerMR, node_added_triggerMR, .auto_add_replicas]
[junit4] 2> 1213208 INFO (ScheduledTrigger-1715-thread-2) [n:127.0.0.1:38684_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1213210 DEBUG (AutoscalingActionExecutor-1716-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"2533be3121b9eT3mumm590e6fkqoi6wywb8tlbw",
[junit4] 2> "source":"node_added_triggerMR",
[junit4] 2> "eventTime":654466631211934,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[654466631211934],
[junit4] 2> "_enqueue_time_":654467741950574,
[junit4] 2> "nodeNames":["127.0.0.1:43653_solr"]}}
[junit4] 2> 1213230 DEBUG (AutoscalingActionExecutor-1716-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_triggerMR after 5000ms
[junit4] 2> 1213230 DEBUG (AutoscalingActionExecutor-1716-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerMR after 5000ms
[junit4] 2> 1213230 DEBUG (AutoscalingActionExecutor-1716-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms
[junit4] 2> 1213230 DEBUG (AutoscalingActionExecutor-1716-thread-1-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 20 ms for event id=2533be3121b9eT3mumm590e6fkqoi6wywb8tlbw
[junit4] 2> 1217578 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.a.TriggerIntegrationTest ====== KILL OVERSEER 2
[junit4] 2> 1217579 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@29a564f6{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 1217579 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=626584319
[junit4] 2> 1217579 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1217580 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@18e01f0e: rootName = solr_38684, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7b54e485
[junit4] 2> 1217598 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1217598 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6d5222ec: rootName = solr_38684, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2fd787d9
[junit4] 2> 1217614 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1217614 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@73e04d3e: rootName = solr_38684, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@242430ac
[junit4] 2> 1217616 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:38684_solr
[junit4] 2> 1217617 INFO (zkCallback-918-thread-1-processing-n:127.0.0.1:43653_solr) [n:127.0.0.1:43653_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1217617 INFO (zkCallback-900-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1217617 INFO (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1217622 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1217635 INFO (zkCallback-890-thread-2-processing-n:127.0.0.1:38684_solr) [n:127.0.0.1:38684_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 1217638 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1217638 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.Overseer Overseer (id=72100484426235910-127.0.0.1:38684_solr-n_0000000001) closing
[junit4] 2> 1217638 INFO (OverseerStateUpdate-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:38684_solr
[junit4] 2> 1217641 DEBUG (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1217650 WARN (OverseerAutoScalingTriggerThread-72100484426235910-127.0.0.1:38684_solr-n_0000000001) [n:127.0.0.1:38684_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1217667 INFO (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44763_solr
[junit4] 2> 1217668 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@72f1060{/solr,null,UNAVAILABLE}
[junit4] 2> 1217668 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1217669 INFO (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.Overseer Overseer (id=72100484426235915-127.0.0.1:44763_solr-n_0000000002) starting
[junit4] 2> 1217706 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1217707 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:44763_solr, 127.0.0.1:43653_solr]
[junit4] 2> 1217724 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44763_solr, 127.0.0.1:43653_solr]
[junit4] 2> 1217724 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerMR instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestEventMarkerAction}], enabled=true}
[junit4] 2> 1217724 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:44763_solr, 127.0.0.1:43653_solr]
[junit4] 2> 1217724 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion -1
[junit4] 2> 1217724 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1217725 INFO (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventMarkerAction init
[junit4] 2> 1217725 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:38684_solr
[junit4] 2> 1217726 INFO (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.TriggerIntegrationTest TestEventMarkerAction init
[junit4] 2> 1217727 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_triggerMR with currently live nodes: 2
[junit4] 2> 1217727 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:38684_solr
[junit4] 2> 1217727 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1217728 DEBUG (ScheduledTrigger-1726-thread-2) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerMR
[junit4] 2> 1217729 DEBUG (ScheduledTrigger-1726-thread-2) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1217742 DEBUG (ScheduledTrigger-1726-thread-3) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1217756 DEBUG (ScheduledTrigger-1726-thread-3) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:38684_solr
[junit4] 2> 1218733 DEBUG (ScheduledTrigger-1726-thread-2) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_triggerMR with currently live nodes: 2
[junit4] 2> 1218733 DEBUG (ScheduledTrigger-1726-thread-2) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:38684_solr]
[junit4] 2> 1218734 DEBUG (ScheduledTrigger-1726-thread-2) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"2533d32ac2336T3mumm590e6fkqoi6wywb8tlby",
[junit4] 2> "source":"node_lost_triggerMR",
[junit4] 2> "eventTime":654472261673782,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[654472261673782],
[junit4] 2> "nodeNames":["127.0.0.1:38684_solr"]}}
[junit4] 2> 1218734 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerMR
[junit4] 2> 1218734 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1218734 DEBUG (ScheduledTrigger-1726-thread-2) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_lost_triggerMR, node_added_triggerMR, .auto_add_replicas]
[junit4] 2> 1218756 INFO (ScheduledTrigger-1726-thread-2) [n:127.0.0.1:44763_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1218769 DEBUG (AutoscalingActionExecutor-1727-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"2533d32ac2336T3mumm590e6fkqoi6wywb8tlby",
[junit4] 2> "source":"node_lost_triggerMR",
[junit4] 2> "eventTime":654472261673782,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[654472261673782],
[junit4] 2> "_enqueue_time_":654473268827819,
[junit4] 2> "nodeNames":["127.0.0.1:38684_solr"]}}
[junit4] 2> 1218769 INFO (TEST-TriggerIntegrationTest.testNodeMarkersRegistration-seed#[8F865384DA767929]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeMarkersRegistration
[junit4] 2> 1218793 DEBUG (AutoscalingActionExecutor-1727-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_triggerMR after 5000ms
[junit4] 2> 1218826 DEBUG (AutoscalingActionExecutor-1727-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerMR after 5000ms
[junit4] 2> 1218826 DEBUG (AutoscalingActionExecutor-1727-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms
[junit4] 2> 1218826 DEBUG (AutoscalingActionExecutor-1727-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 56 ms for event id=2533d32ac2336T3mumm590e6fkqoi6wywb8tlby
[junit4] 2> 1218865 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testListeners
[junit4] 2> 1218866 DEBUG (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1218866 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1218879 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 6
[junit4] 2> 1218897 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1218897 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1218897 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 1219100 INFO (qtp243726826-5800) [n:127.0.0.1:43653_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1219192 INFO (qtp1141431223-5763) [n:127.0.0.1:44763_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=53
[junit4] 2> 1219193 DEBUG (qtp243726826-5799) [n:127.0.0.1:43653_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1219205 INFO (qtp243726826-5799) [n:127.0.0.1:43653_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=142
[junit4] 2> 1219205 DEBUG (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1219206 DEBUG (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44763_solr, 127.0.0.1:43653_solr]
[junit4] 2> 1219206 DEBUG (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 1219206 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1219222 INFO (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1219223 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1219223 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 1219269 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1219269 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1219278 INFO (qtp243726826-5794) [n:127.0.0.1:43653_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1219280 INFO (qtp1141431223-5760) [n:127.0.0.1:44763_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1219281 DEBUG (qtp243726826-5797) [n:127.0.0.1:43653_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1219281 INFO (qtp243726826-5797) [n:127.0.0.1:43653_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=58
[junit4] 2> 1219295 INFO (qtp243726826-5800) [n:127.0.0.1:43653_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1219296 INFO (qtp1141431223-5763) [n:127.0.0.1:44763_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1219296 DEBUG (qtp243726826-5799) [n:127.0.0.1:43653_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1219302 DEBUG (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 1219314 INFO (qtp243726826-5799) [n:127.0.0.1:43653_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 1219316 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1219318 DEBUG (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44763_solr, 127.0.0.1:43653_solr]
[junit4] 2> 1219318 DEBUG (zkCallback-908-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 1219355 DEBUG (zkCallback-908-thread-2-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 1219375 DEBUG (zkCallback-908-thread-2-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:44763_solr, 127.0.0.1:43653_solr]
[junit4] 2> 1219375 DEBUG (zkCallback-908-thread-2-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestDummyAction}], enabled=true}
[junit4] 2> 1219375 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 1219376 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1219376 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1219376 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1219391 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1219392 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1ba75c6f{/solr,null,AVAILABLE}
[junit4] 2> 1219392 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3d18ff59{SSL,[ssl, http/1.1]}{127.0.0.1:43215}
[junit4] 2> 1219392 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.Server Started @1229717ms
[junit4] 2> 1219392 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43215}
[junit4] 2> 1219392 ERROR (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1219424 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1219424 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1219424 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1219424 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-11T02:44:33.021Z
[junit4] 2> 1219432 DEBUG (OverseerAutoScalingTriggerThread-72100484426235915-127.0.0.1:44763_solr-n_0000000002) [n:127.0.0.1:44763_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 1219449 INFO (zkConnectionManagerCallback-923-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1219449 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1219520 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:44676/solr
[junit4] 2> 1219538 INFO (zkConnectionManagerCallback-927-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1219680 INFO (zkConnectionManagerCallback-929-thread-1-processing-n:127.0.0.1:43215_solr) [n:127.0.0.1:43215_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1219702 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [n:127.0.0.1:43215_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1219738 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [n:127.0.0.1:43215_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1219775 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [n:127.0.0.1:43215_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43215_solr
[junit4] 2> 1219794 INFO (zkCallback-908-thread-2-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1219806 INFO (zkCallback-918-thread-1-processing-n:127.0.0.1:43653_solr) [n:127.0.0.1:43653_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1219810 INFO (zkCallback-900-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1219884 INFO (zkCallback-928-thread-1-processing-n:127.0.0.1:43215_solr) [n:127.0.0.1:43215_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1220315 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1220315 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1220315 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:43215_solr at time 654474849311072
[junit4] 2> 1220422 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [n:127.0.0.1:43215_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43215.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1220468 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [n:127.0.0.1:43215_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43215.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1220468 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [n:127.0.0.1:43215_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43215.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4dd09833
[junit4] 2> 1220482 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [n:127.0.0.1:43215_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_8F865384DA767929-001/tempDir-001/node5/.
[junit4] 2> 1221352 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1221352 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1222363 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1222364 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1223364 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerL
[junit4] 2> 1223364 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1223364 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerL firing registered processor for nodes: [127.0.0.1:43215_solr] added at times [654474849311072], now=654477898527467
[junit4] 2> 1223364 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"2533dcce85960T3mumm590e6fkqoi6wywb8tlc0",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":654474849311072,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[654474849311072],
[junit4] 2> "nodeNames":["127.0.0.1:43215_solr"]}}
[junit4] 2> 1223364 DEBUG (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [node_added_triggerL]
[junit4] 2> 1223365 INFO (ScheduledTrigger-1726-thread-1) [n:127.0.0.1:44763_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1223366 DEBUG (AutoscalingActionExecutor-1727-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"2533dcce85960T3mumm590e6fkqoi6wywb8tlc0",
[junit4] 2> "source":"node_added_triggerL",
[junit4] 2> "eventTime":654474849311072,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[654474849311072],
[junit4] 2> "_enqueue_time_":654477898739904,
[junit4] 2> "nodeNames":["127.0.0.1:43215_solr"]}}
[junit4] 2> 1223389 DEBUG (AutoscalingActionExecutor-1727-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerL after 5000ms
[junit4] 2> 1223389 DEBUG (AutoscalingActionExecutor-1727-thread-1-processing-n:127.0.0.1:44763_solr) [n:127.0.0.1:44763_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 22 ms for event id=2533dcce85960T3mumm590e6fkqoi6wywb8tlc0
[junit4] 2> 1225370 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:27:37+05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1225371 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1225371 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1225371 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1225371 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@30a105a8{/solr,null,AVAILABLE}
[junit4] 2> 1225372 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@59ea5de1{SSL,[ssl, http/1.1]}{127.0.0.1:38244}
[junit4] 2> 1225372 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.e.j.s.Server Started @1235696ms
[junit4] 2> 1225372 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38244}
[junit4] 2> 1225372 ERROR (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1225373 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 1225373 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1225373 INFO (TEST-TriggerIntegrationTest.testListeners-seed#[8F865384DA767929]) [ ] o.a.
[...truncated too long message...]
eReader.java:704)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:703)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$900(ZkStateReader.java:72)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1090)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1085)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:269)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 4385516 INFO (jetty-closer-4004-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@e61d98c{/solr,null,UNAVAILABLE}
[junit4] 2> 4385516 INFO (jetty-closer-4004-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 4385517 ERROR (SUITE-CollectionsAPIDistributedZkTest-seed#[8F865384DA767929]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4385517 INFO (SUITE-CollectionsAPIDistributedZkTest-seed#[8F865384DA767929]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41745 41745
[junit4] 2> 4385518 INFO (Thread-6962) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41745 41745
[junit4] 2> 4385519 WARN (Thread-6962) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/acollectionafterbaddelete/terms/shard1
[junit4] 2> 4 /solr/aliases.json
[junit4] 2> 3 /solr/configs/conf
[junit4] 2> 2 /solr/collections/created_and_deleted/terms/shard1
[junit4] 2> 2 /solr/collections/awhollynewcollection_0/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 17 /solr/collections/acollectionafterbaddelete/state.json
[junit4] 2> 5 /solr/collections/created_and_deleted/state.json
[junit4] 2> 4 /solr/clusterstate.json
[junit4] 2> 4 /solr/clusterprops.json
[junit4] 2> 4 /solr/collections/awhollynewcollection_0/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 123 /solr/collections
[junit4] 2> 4 /solr/live_nodes
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.api.collections.CollectionsAPIDistributedZkTest_8F865384DA767929-001
[junit4] 2> Feb 11, 2018 3:37:19 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=285, maxMBSortInHeap=6.216128820735833, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@5337ac4e), locale=ko-KR, timezone=Europe/Amsterdam
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=226089472,total=519045120
[junit4] 2> NOTE: All tests run in this JVM: [TestSolrDeletionPolicy1, CurrencyFieldTypeTest, TestRequestStatusCollectionAPI, TestIndexSearcher, AutoscalingHistoryHandlerTest, TestDistributedSearch, CdcrUpdateLogTest, TestCSVLoader, TestSolrConfigHandler, BasicDistributedZkTest, TestFileDictionaryLookup, TestTlogReplica, SimpleCollectionCreateDeleteTest, SolrIndexConfigTest, TestLuceneIndexBackCompat, SpellingQueryConverterTest, TestLegacyFieldCache, UUIDFieldTest, TestJavabinTupleStreamParser, TestDistribDocBasedVersion, TestNoOpRegenerator, TestDocumentBuilder, TestStressLucene, SolrCoreCheckLockOnStartupTest, TestHighlightDedupGrouping, TestCollationFieldDocValues, TestExecutePlanAction, DeleteShardTest, SolrJmxReporterTest, TestPivotHelperCode, HDFSCollectionsAPITest, DirectUpdateHandlerTest, DistributedDebugComponentTest, ExitableDirectoryReaderTest, HdfsSyncSliceTest, TestLegacyField, HdfsDirectoryTest, SystemLogListenerTest, UUIDUpdateProcessorFallbackTest, SmileWriterTest, TestConfigSetsAPIZkFailure, SolrXmlInZkTest, TestMaxTokenLenTokenizer, PreAnalyzedUpdateProcessorTest, TestMacroExpander, TestExportWriter, HLLSerializationTest, DeleteLastCustomShardedReplicaTest, VMParamsZkACLAndCredentialsProvidersTest, TestSmileRequest, MBeansHandlerTest, BlockJoinFacetSimpleTest, HighlighterConfigTest, JvmMetricsTest, TestManagedStopFilterFactory, TestHashPartitioner, TestSimDistributedQueue, LargeFieldTest, TestCloudInspectUtil, ZkNodePropsTest, DistribDocExpirationUpdateProcessorTest, UpdateLogTest, HighlighterMaxOffsetTest, PluginInfoTest, HdfsAutoAddReplicasIntegrationTest, HealthCheckHandlerTest, DistributedQueryElevationComponentTest, DistributedVersionInfoTest, PKIAuthenticationIntegrationTest, HdfsUnloadDistributedZkTest, HdfsRecoverLeaseTest, ActionThrottleTest, TestBackupRepositoryFactory, TestClassicSimilarityFactory, SpellCheckCollatorTest, ChaosMonkeySafeLeaderWithPullReplicasTest, ZkSolrClientTest, TestNumericRangeQuery64, TestRuleBasedAuthorizationPlugin, TestSort, SolrMetricManagerTest, ReplaceNodeTest, TestCSVResponseWriter, TestSolrCoreSnapshots, TestStressCloudBlindAtomicUpdates, CloudExitableDirectoryReaderTest, LeaderFailureAfterFreshStartTest, TestBulkSchemaConcurrent, TestConfigsApi, StatsComponentTest, TestPseudoReturnFields, MigrateRouteKeyTest, TestChildDocTransformer, TestOmitPositions, TestConfigSetProperties, TestDistribIDF, SpatialHeatmapFacetsTest, AutoScalingHandlerTest, TestLockTree, CdcrBidirectionalTest, TestUtilizeNode, TestLMDirichletSimilarityFactory, UninvertDocValuesMergePolicyTest, RecoveryAfterSoftCommitTest, DistributedFacetPivotLargeTest, TestSolrFieldCacheBean, TestLegacyTerms, BadCopyFieldTest, TestJoin, OutputWriterTest, RequestHandlersTest, SuggesterWFSTTest, TestDynamicFieldCollectionResource, TestLuceneMatchVersion, TestFieldCacheSortRandom, TestManagedSchemaAPI, ShardRoutingTest, TestClusterStateProvider, SolrShardReporterTest, SpellCheckCollatorWithCollapseTest, TestHdfsCloudBackupRestore, TermVectorComponentDistributedTest, MoveReplicaHDFSTest, TestComponentsName, SharedFSAutoReplicaFailoverTest, ChaosMonkeyNothingIsSafeTest, NodeMutatorTest, TestRandomFaceting, ZkCLITest, ZkControllerTest, TestReload, TestCoreContainer, SimpleFacetsTest, TestSolr4Spatial, SolrCmdDistributorTest, BadIndexSchemaTest, BasicFunctionalityTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, SignatureUpdateProcessorFactoryTest, TestExtendedDismaxParser, SuggesterFSTTest, CoreAdminHandlerTest, SolrRequestParserTest, DocValuesTest, SuggesterTSTTest, SuggesterTest, SpatialFilterTest, PolyFieldTest, NoCacheHeaderTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, TestUpdate, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, TestRemoteStreaming, SolrInfoBeanTest, DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, LukeRequestHandlerTest, DocumentAnalysisRequestHandlerTest, TestQueryTypes, XmlUpdateRequestHandlerTest, TermVectorComponentTest, FieldAnalysisRequestHandlerTest, IndexSchemaRuntimeFieldTest, SolrPluginUtilsTest, TestJmxIntegration, UpdateRequestProcessorFactoryTest, TestAnalyzedSuggestions, BinaryUpdateRequestHandlerTest, CSVRequestHandlerTest, TestLFUCache, TestMergePolicyConfig, TestSolrDeletionPolicy2, TestDocSet, TestBinaryField, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, TestConfig, ExternalFileFieldSortTest, TestPhraseSuggestions, TestCharFilters, SynonymTokenizerTest, TestXIncludeConfig, EchoParamsTest, TestSweetSpotSimilarityFactory, TestPerFieldSimilarity, TestIBSimilarityFactory, TestBM25SimilarityFactory, TestFastWriter, ResourceLoaderTest, TestFastOutputStream, OpenExchangeRatesOrgProviderTest, URLClassifyProcessorTest, TestFastLRUCache, DateMathParserTest, TestSystemIdResolver, PrimUtilsTest, TestSuggestSpellingConverter, RAMDirectoryFactoryTest, ClusterStateTest, TestSolrJ, TestLRUCache, TestUtils, SystemInfoHandlerTest, DistributedMLTComponentTest, TestRTGBase, CursorPagingTest, DistributedIntervalFacetingTest, TestCrossCoreJoin, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, AddReplicaTest, AssignBackwardCompatibilityTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, ConnectionManagerTest, CreateCollectionCleanupTest, DeleteInactiveReplicaTest, DistributedQueueTest, LIRRollingUpdatesTest, OverseerCollectionConfigSetProcessorTest, OverseerRolesTest, OverseerStatusTest, RemoteQueryErrorTest, ReplaceNodeNoTargetTest, RestartWhileUpdatingTest, RollingRestartTest, TestCloudPseudoReturnFields, TestCloudRecovery, TestPrepRecovery, TestRandomFlRTGCloud, TestRequestForwarding, TestSegmentSorting, TriLevelCompositeIdRoutingTest, CollectionTooManyReplicasTest, CollectionsAPIDistributedZkTest]
[junit4] Completed [580/776 (2!)] on J0 in 132.62s, 19 tests, 1 error <<< FAILURES!
[...truncated 48905 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2307 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2307/
3 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
Error Message:
Waiting for collection testSimple1 null Live Nodes: [127.0.0.1:36009_solr, 127.0.0.1:44350_solr] Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/17)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node2":{ "dataDir":"hdfs://localhost:36294/data/testSimple1/core_node2/data/", "base_url":"http://127.0.0.1:44350/solr", "node_name":"127.0.0.1:44350_solr", "type":"NRT", "ulogDir":"hdfs://localhost:36294/data/testSimple1/core_node2/data/tlog", "core":"testSimple1_shard1_replica_n1", "shared_storage":"true", "state":"active"}, "core_node6":{ "dataDir":"hdfs://localhost:36294/data/testSimple1/core_node6/data/", "base_url":"http://127.0.0.1:44350/solr", "node_name":"127.0.0.1:44350_solr", "type":"NRT", "ulogDir":"hdfs://localhost:36294/data/testSimple1/core_node6/data/tlog", "core":"testSimple1_shard1_replica_n3", "shared_storage":"true", "state":"active", "leader":"true"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node7":{ "dataDir":"hdfs://localhost:36294/data/testSimple1/core_node7/data/", "base_url":"http://127.0.0.1:33278/solr", "node_name":"127.0.0.1:33278_solr", "type":"NRT", "ulogDir":"hdfs://localhost:36294/data/testSimple1/core_node7/data/tlog", "core":"testSimple1_shard2_replica_n4", "shared_storage":"true", "state":"down"}, "core_node8":{ "dataDir":"hdfs://localhost:36294/data/testSimple1/core_node8/data/", "base_url":"http://127.0.0.1:44350/solr", "node_name":"127.0.0.1:44350_solr", "type":"NRT", "ulogDir":"hdfs://localhost:36294/data/testSimple1/core_node8/data/tlog", "core":"testSimple1_shard2_replica_n5", "shared_storage":"true", "state":"active", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"true", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple1
null
Live Nodes: [127.0.0.1:36009_solr, 127.0.0.1:44350_solr]
Last available state: DocCollection(testSimple1//collections/testSimple1/state.json/17)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node2":{
"dataDir":"hdfs://localhost:36294/data/testSimple1/core_node2/data/",
"base_url":"http://127.0.0.1:44350/solr",
"node_name":"127.0.0.1:44350_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost:36294/data/testSimple1/core_node2/data/tlog",
"core":"testSimple1_shard1_replica_n1",
"shared_storage":"true",
"state":"active"},
"core_node6":{
"dataDir":"hdfs://localhost:36294/data/testSimple1/core_node6/data/",
"base_url":"http://127.0.0.1:44350/solr",
"node_name":"127.0.0.1:44350_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost:36294/data/testSimple1/core_node6/data/tlog",
"core":"testSimple1_shard1_replica_n3",
"shared_storage":"true",
"state":"active",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node7":{
"dataDir":"hdfs://localhost:36294/data/testSimple1/core_node7/data/",
"base_url":"http://127.0.0.1:33278/solr",
"node_name":"127.0.0.1:33278_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost:36294/data/testSimple1/core_node7/data/tlog",
"core":"testSimple1_shard2_replica_n4",
"shared_storage":"true",
"state":"down"},
"core_node8":{
"dataDir":"hdfs://localhost:36294/data/testSimple1/core_node8/data/",
"base_url":"http://127.0.0.1:44350/solr",
"node_name":"127.0.0.1:44350_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost:36294/data/testSimple1/core_node8/data/tlog",
"core":"testSimple1_shard2_replica_n5",
"shared_storage":"true",
"state":"active",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"true",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([B2B0A5BDC44833E9:8A038143E3BBE738]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:103)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testCooldown
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([B2B0A5BDC44833E9:830EC859BAE2461B]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testCooldown(TriggerIntegrationTest.java:1177)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([B2B0A5BDC44833E9:DF4C01407E00CCEE]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:244)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12165 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/init-core-data-001
[junit4] 2> 2781 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 2782 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2784 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2785 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2791 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-001
[junit4] 2> 2799 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2801 INFO (Thread-2) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2802 INFO (Thread-2) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2900 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:37934
[junit4] 2> 2971 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Exception causing close of session 0x0: ZooKeeperServer not running
[junit4] 2> 3027 ERROR (Thread-2) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 4728 INFO (zkConnectionManagerCallback-12-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 4943 INFO (jetty-launcher-9-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T19:27:37-02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 4943 INFO (jetty-launcher-9-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T19:27:37-02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 4943 INFO (jetty-launcher-9-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T19:27:37-02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 4964 INFO (jetty-launcher-9-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4964 INFO (jetty-launcher-9-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4968 INFO (jetty-launcher-9-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 4984 INFO (jetty-launcher-9-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4992 INFO (jetty-launcher-9-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4992 INFO (jetty-launcher-9-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 4998 INFO (jetty-launcher-9-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 4998 INFO (jetty-launcher-9-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 4998 INFO (jetty-launcher-9-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 5001 INFO (jetty-launcher-9-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4dcda159{/solr,null,AVAILABLE}
[junit4] 2> 5005 INFO (jetty-launcher-9-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@199637d2{/solr,null,AVAILABLE}
[junit4] 2> 5002 INFO (jetty-launcher-9-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@79e365d7{/solr,null,AVAILABLE}
[junit4] 2> 5025 INFO (jetty-launcher-9-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1765a595{HTTP/1.1,[http/1.1]}{127.0.0.1:33278}
[junit4] 2> 5025 INFO (jetty-launcher-9-thread-1) [ ] o.e.j.s.Server Started @7224ms
[junit4] 2> 5026 INFO (jetty-launcher-9-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3ae0f1d6{HTTP/1.1,[http/1.1]}{127.0.0.1:44350}
[junit4] 2> 5026 INFO (jetty-launcher-9-thread-2) [ ] o.e.j.s.Server Started @7225ms
[junit4] 2> 5026 INFO (jetty-launcher-9-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44350}
[junit4] 2> 5026 INFO (jetty-launcher-9-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33278}
[junit4] 2> 5025 INFO (jetty-launcher-9-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7f6ea03d{HTTP/1.1,[http/1.1]}{127.0.0.1:36009}
[junit4] 2> 5027 INFO (jetty-launcher-9-thread-3) [ ] o.e.j.s.Server Started @7226ms
[junit4] 2> 5035 INFO (jetty-launcher-9-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36009}
[junit4] 2> 5044 ERROR (jetty-launcher-9-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5044 INFO (jetty-launcher-9-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 5044 INFO (jetty-launcher-9-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5044 INFO (jetty-launcher-9-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 5046 ERROR (jetty-launcher-9-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5046 INFO (jetty-launcher-9-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 5054 INFO (jetty-launcher-9-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5054 INFO (jetty-launcher-9-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 5054 ERROR (jetty-launcher-9-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 5054 INFO (jetty-launcher-9-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 5055 INFO (jetty-launcher-9-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 5055 INFO (jetty-launcher-9-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 5059 INFO (jetty-launcher-9-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T20:34:35.265Z
[junit4] 2> 5060 INFO (jetty-launcher-9-thread-3) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 5060 INFO (jetty-launcher-9-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T20:34:35.266Z
[junit4] 2> 5061 INFO (jetty-launcher-9-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-10T20:34:35.257Z
[junit4] 2> 5076 INFO (zkConnectionManagerCallback-14-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5086 INFO (zkConnectionManagerCallback-16-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5104 INFO (jetty-launcher-9-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5105 INFO (jetty-launcher-9-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5119 INFO (zkConnectionManagerCallback-18-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5134 INFO (jetty-launcher-9-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 5182 INFO (jetty-launcher-9-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37934/solr
[junit4] 2> 5198 INFO (jetty-launcher-9-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37934/solr
[junit4] 2> 5210 INFO (zkConnectionManagerCallback-22-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5222 INFO (jetty-launcher-9-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:37934/solr
[junit4] 2> 5220 INFO (zkConnectionManagerCallback-26-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5246 INFO (zkConnectionManagerCallback-30-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5282 INFO (zkConnectionManagerCallback-32-thread-1-processing-n:127.0.0.1:33278_solr) [n:127.0.0.1:33278_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5282 INFO (zkConnectionManagerCallback-34-thread-1-processing-n:127.0.0.1:44350_solr) [n:127.0.0.1:44350_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5293 INFO (zkConnectionManagerCallback-36-thread-1-processing-n:127.0.0.1:36009_solr) [n:127.0.0.1:36009_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 5642 INFO (jetty-launcher-9-thread-2) [n:127.0.0.1:44350_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 5642 INFO (jetty-launcher-9-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 5649 INFO (jetty-launcher-9-thread-3) [n:127.0.0.1:36009_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 5649 INFO (jetty-launcher-9-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33278_solr
[junit4] 2> 5656 INFO (jetty-launcher-9-thread-2) [n:127.0.0.1:44350_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44350_solr
[junit4] 2> 5658 INFO (jetty-launcher-9-thread-3) [n:127.0.0.1:36009_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36009_solr
[junit4] 2> 5666 INFO (jetty-launcher-9-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.Overseer Overseer (id=73273076056260615-127.0.0.1:33278_solr-n_0000000000) starting
[junit4] 2> 5739 INFO (zkCallback-31-thread-1-processing-n:127.0.0.1:33278_solr) [n:127.0.0.1:33278_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 5746 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:36009_solr) [n:127.0.0.1:36009_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 5746 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:44350_solr) [n:127.0.0.1:44350_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 5852 INFO (jetty-launcher-9-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33278_solr
[junit4] 2> 5855 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:36009_solr) [n:127.0.0.1:36009_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 5855 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:44350_solr) [n:127.0.0.1:44350_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 5855 INFO (zkCallback-31-thread-1-processing-n:127.0.0.1:33278_solr) [n:127.0.0.1:33278_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 6042 DEBUG (OverseerAutoScalingTriggerThread-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 6054 DEBUG (OverseerAutoScalingTriggerThread-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 6097 DEBUG (OverseerAutoScalingTriggerThread-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:33278_solr, 127.0.0.1:36009_solr, 127.0.0.1:44350_solr]
[junit4] 2> 6097 DEBUG (OverseerAutoScalingTriggerThread-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 6097 DEBUG (OverseerAutoScalingTriggerThread-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 6100 INFO (jetty-launcher-9-thread-3) [n:127.0.0.1:36009_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36009.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 6101 INFO (jetty-launcher-9-thread-3) [n:127.0.0.1:36009_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36009.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 6101 INFO (jetty-launcher-9-thread-3) [n:127.0.0.1:36009_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36009.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 6102 INFO (jetty-launcher-9-thread-2) [n:127.0.0.1:44350_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44350.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 6103 INFO (jetty-launcher-9-thread-2) [n:127.0.0.1:44350_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44350.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 6103 INFO (jetty-launcher-9-thread-2) [n:127.0.0.1:44350_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44350.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 6105 DEBUG (OverseerAutoScalingTriggerThread-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 6106 DEBUG (OverseerAutoScalingTriggerThread-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 6107 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 6126 INFO (jetty-launcher-9-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33278.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 6126 INFO (jetty-launcher-9-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33278.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 6126 INFO (jetty-launcher-9-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33278.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 6129 INFO (jetty-launcher-9-thread-2) [n:127.0.0.1:44350_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-001/node2/.
[junit4] 2> 6143 INFO (jetty-launcher-9-thread-3) [n:127.0.0.1:36009_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-001/node3/.
[junit4] 2> 6144 INFO (jetty-launcher-9-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-001/node1/.
[junit4] 2> 6317 INFO (zkConnectionManagerCallback-44-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 6327 INFO (zkConnectionManagerCallback-48-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 6330 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 6332 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:37934/solr ready
[junit4] 2> 6801 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.h.u.NativeCodeLoader Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 7122 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 7441 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 7578 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.m.log Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
[junit4] 2> 7598 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 7861 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 7895 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_localhost_34958_hdfs____wb837p/webapp
[junit4] 2> 8123 DEBUG (ScheduledTrigger-26-thread-2) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 8957 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:34958
[junit4] 2> 9123 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 9659 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 9663 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 9672 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_33621_datanode____mez858/webapp
[junit4] 2> 10075 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33621
[junit4] 2> 10123 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 10423 WARN (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 10425 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 10447 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.m.log Extract jar:file:/x1/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_33040_datanode____30jtpv/webapp
[junit4] 2> 10950 INFO (SUITE-HdfsAutoAddReplicasIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33040
[junit4] 2> 11108 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to localhost/127.0.0.1:36294) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 11126 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 11276 ERROR (DataNode: [[[DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to localhost/127.0.0.1:36294) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 11329 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x41e435903f41b0: from storage DS-bd9cf523-437c-4727-9ef2-c6d37f7dd53b node DatanodeRegistration(127.0.0.1:38150, datanodeUuid=8061efb5-9d6b-4956-b945-f0a1abdd9b26, infoPort=34095, infoSecurePort=0, ipcPort=33135, storageInfo=lv=-56;cid=testClusterID;nsid=897729064;c=0), blocks: 0, hasStaleStorage: true, processing time: 2 msecs
[junit4] 2> 11330 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x41e435903f4192: from storage DS-5c96832f-dda4-4b6e-888e-5726ec4a6272 node DatanodeRegistration(127.0.0.1:35828, datanodeUuid=f52b348a-ddb2-44e2-8bfd-34587be89595, infoPort=36094, infoSecurePort=0, ipcPort=60746, storageInfo=lv=-56;cid=testClusterID;nsid=897729064;c=0), blocks: 0, hasStaleStorage: true, processing time: 1 msecs
[junit4] 2> 11330 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x41e435903f41b0: from storage DS-c4e53337-e2a4-442d-988a-cec3c0754a21 node DatanodeRegistration(127.0.0.1:38150, datanodeUuid=8061efb5-9d6b-4956-b945-f0a1abdd9b26, infoPort=34095, infoSecurePort=0, ipcPort=33135, storageInfo=lv=-56;cid=testClusterID;nsid=897729064;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 11330 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x41e435903f4192: from storage DS-7ed81f3f-67c4-44cb-acf9-6541cb4555d9 node DatanodeRegistration(127.0.0.1:35828, datanodeUuid=f52b348a-ddb2-44e2-8bfd-34587be89595, infoPort=36094, infoSecurePort=0, ipcPort=60746, storageInfo=lv=-56;cid=testClusterID;nsid=897729064;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 12126 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 12366 INFO (TEST-HdfsAutoAddReplicasIntegrationTest.testSimple-seed#[B2B0A5BDC44833E9]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSimple
[junit4] 2> 12679 INFO (qtp1774229243-38) [n:127.0.0.1:36009_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=2&autoAddReplicas=true&name=testSimple1&nrtReplicas=2&action=CREATE&numShards=2&createNodeSet=127.0.0.1:36009_solr,127.0.0.1:44350_solr&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 12705 INFO (OverseerThreadFactory-29-thread-1-processing-n:127.0.0.1:33278_solr) [n:127.0.0.1:33278_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testSimple1
[junit4] 2> 12861 INFO (OverseerStateUpdate-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testSimple1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36009/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 12886 INFO (OverseerStateUpdate-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testSimple1_shard1_replica_n3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44350/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 12889 INFO (OverseerStateUpdate-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testSimple1_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36009/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 12894 INFO (OverseerStateUpdate-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testSimple1_shard2_replica_n5",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44350/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 13105 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=testSimple1_shard1_replica_n3&action=CREATE&numShards=2&collection=testSimple1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 13107 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=testSimple1_shard2_replica_n4&action=CREATE&numShards=2&collection=testSimple1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 13110 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=testSimple1_shard1_replica_n1&action=CREATE&numShards=2&collection=testSimple1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 13111 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 13122 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 13124 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=testSimple1_shard2_replica_n5&action=CREATE&numShards=2&collection=testSimple1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 13126 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 13273 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:44350_solr) [n:127.0.0.1:44350_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 13277 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:36009_solr) [n:127.0.0.1:36009_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 14127 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 14173 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 14183 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 14190 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 14239 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 14272 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.s.IndexSchema [testSimple1_shard1_replica_n3] Schema name=minimal
[junit4] 2> 14285 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.s.IndexSchema [testSimple1_shard2_replica_n4] Schema name=minimal
[junit4] 2> 14289 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.s.IndexSchema [testSimple1_shard2_replica_n5] Schema name=minimal
[junit4] 2> 14307 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.s.IndexSchema [testSimple1_shard1_replica_n1] Schema name=minimal
[junit4] 2> 14320 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 14320 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'testSimple1_shard2_replica_n4' using configuration from collection testSimple1, trusted=true
[junit4] 2> 14321 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 14321 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'testSimple1_shard1_replica_n1' using configuration from collection testSimple1, trusted=true
[junit4] 2> 14338 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36009.solr.core.testSimple1.shard2.replica_n4' (registry 'solr.core.testSimple1.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 14346 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36009.solr.core.testSimple1.shard1.replica_n1' (registry 'solr.core.testSimple1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 14347 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 14347 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'testSimple1_shard1_replica_n3' using configuration from collection testSimple1, trusted=true
[junit4] 2> 14350 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 14350 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'testSimple1_shard2_replica_n5' using configuration from collection testSimple1, trusted=true
[junit4] 2> 14351 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44350.solr.core.testSimple1.shard2.replica_n5' (registry 'solr.core.testSimple1.shard2.replica_n5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 14354 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44350.solr.core.testSimple1.shard1.replica_n3' (registry 'solr.core.testSimple1.shard1.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5cc83a41
[junit4] 2> 14354 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:36294/data
[junit4] 2> 14354 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 14354 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 14355 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.c.SolrCore [[testSimple1_shard2_replica_n5] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-001/node2/testSimple1_shard2_replica_n5], dataDir=[hdfs://localhost:36294/data/testSimple1/core_node8/data/]
[junit4] 2> 14359 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:36294/data
[junit4] 2> 14365 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 14365 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 14365 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.SolrCore [[testSimple1_shard1_replica_n3] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-001/node2/testSimple1_shard1_replica_n3], dataDir=[hdfs://localhost:36294/data/testSimple1/core_node6/data/]
[junit4] 2> 14369 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node6/data/snapshot_metadata
[junit4] 2> 14370 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:36294/data
[junit4] 2> 14370 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 14370 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 14370 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.c.SolrCore [[testSimple1_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-001/node3/testSimple1_shard1_replica_n1], dataDir=[hdfs://localhost:36294/data/testSimple1/core_node2/data/]
[junit4] 2> 14371 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node8/data/snapshot_metadata
[junit4] 2> 14372 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node2/data/snapshot_metadata
[junit4] 2> 14378 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://localhost:36294/data
[junit4] 2> 14378 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 14378 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 14378 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.SolrCore [[testSimple1_shard2_replica_n4] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest_B2B0A5BDC44833E9-001/tempDir-001/node3/testSimple1_shard2_replica_n4], dataDir=[hdfs://localhost:36294/data/testSimple1/core_node7/data/]
[junit4] 2> 14380 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node7/data/snapshot_metadata
[junit4] 2> 14516 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node7/data
[junit4] 2> 14518 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node2/data
[junit4] 2> 14522 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node8/data
[junit4] 2> 14522 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node6/data
[junit4] 2> 14651 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node7/data/index
[junit4] 2> 14653 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node8/data/index
[junit4] 2> 14658 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node6/data/index
[junit4] 2> 14662 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://localhost:36294/data/testSimple1/core_node2/data/index
[junit4] 2> 15127 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 15261 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35828 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-bd9cf523-437c-4727-9ef2-c6d37f7dd53b:NORMAL:127.0.0.1:38150|RBW], ReplicaUC[[DISK]DS-7ed81f3f-67c4-44cb-acf9-6541cb4555d9:NORMAL:127.0.0.1:35828|RBW]]} size 0
[junit4] 2> 15263 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38150 is added to blk_1073741828_1004{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-5c96832f-dda4-4b6e-888e-5726ec4a6272:NORMAL:127.0.0.1:35828|RBW], ReplicaUC[[DISK]DS-c4e53337-e2a4-442d-988a-cec3c0754a21:NORMAL:127.0.0.1:38150|RBW]]} size 69
[junit4] 2> 15263 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35828 is added to blk_1073741825_1001{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-5c96832f-dda4-4b6e-888e-5726ec4a6272:NORMAL:127.0.0.1:35828|RBW], ReplicaUC[[DISK]DS-bd9cf523-437c-4727-9ef2-c6d37f7dd53b:NORMAL:127.0.0.1:38150|RBW]]} size 69
[junit4] 2> 15263 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35828 is added to blk_1073741828_1004 size 69
[junit4] 2> 15263 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:35828 is added to blk_1073741826_1002{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-bd9cf523-437c-4727-9ef2-c6d37f7dd53b:NORMAL:127.0.0.1:38150|RBW], ReplicaUC[[DISK]DS-7ed81f3f-67c4-44cb-acf9-6541cb4555d9:NORMAL:127.0.0.1:35828|RBW]]} size 69
[junit4] 2> 15264 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38150 is added to blk_1073741825_1001 size 69
[junit4] 2> 15264 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38150 is added to blk_1073741826_1002 size 69
[junit4] 2> 15276 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:38150 is added to blk_1073741827_1003 size 69
[junit4] 2> 15384 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 15385 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 15385 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 15406 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 15408 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 15495 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@b478360[testSimple1_shard2_replica_n4] main]
[junit4] 2> 15500 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 15501 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 15504 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 15507 INFO (searcherExecutor-35-thread-1-processing-n:127.0.0.1:36009_solr x:testSimple1_shard2_replica_n4 s:shard2 c:testSimple1 r:core_node7) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.SolrCore [testSimple1_shard2_replica_n4] Registered new searcher Searcher@b478360[testSimple1_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 15511 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592047578090831872
[junit4] 2> 15568 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9998ms
[junit4] 2> 15570 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:44350_solr) [n:127.0.0.1:44350_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 15571 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:36009_solr) [n:127.0.0.1:36009_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 15826 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 15826 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 15826 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 15866 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 15874 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 15874 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 15891 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 15887 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 15894 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 15894 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 15892 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 15916 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 15916 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 15926 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 15926 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 15969 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c9009c[testSimple1_shard1_replica_n1] main]
[junit4] 2> 15975 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.s.SolrIndexSearcher Opening [Searcher@bd462c0[testSimple1_shard2_replica_n5] main]
[junit4] 2> 15972 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 15976 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 15987 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 15989 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 15988 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 15990 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 16000 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@c4e27e9[testSimple1_shard1_replica_n3] main]
[junit4] 2> 16002 INFO (searcherExecutor-36-thread-1-processing-n:127.0.0.1:36009_solr x:testSimple1_shard1_replica_n1 s:shard1 c:testSimple1 r:core_node2) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.c.SolrCore [testSimple1_shard1_replica_n1] Registered new searcher Searcher@5c9009c[testSimple1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 16002 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592047578605682688
[junit4] 2> 16003 INFO (searcherExecutor-38-thread-1-processing-n:127.0.0.1:44350_solr x:testSimple1_shard2_replica_n5 s:shard2 c:testSimple1 r:core_node8) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.c.SolrCore [testSimple1_shard2_replica_n5] Registered new searcher Searcher@bd462c0[testSimple1_shard2_replica_n5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 16003 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592047578606731264
[junit4] 2> 16010 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 16010 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 16021 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 16022 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592047578626654208
[junit4] 2> 16026 INFO (searcherExecutor-37-thread-1-processing-n:127.0.0.1:44350_solr x:testSimple1_shard1_replica_n3 s:shard1 c:testSimple1 r:core_node6) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.SolrCore [testSimple1_shard1_replica_n3] Registered new searcher Searcher@c4e27e9[testSimple1_shard1_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 16038 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 16038 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 16038 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44350/solr/testSimple1_shard1_replica_n3/
[junit4] 2> 16043 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=testSimple1_shard1_replica_n3 url=http://127.0.0.1:44350/solr START replicas=[http://127.0.0.1:36009/solr/testSimple1_shard1_replica_n1/] nUpdates=100
[junit4] 2> 16074 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 16074 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 16074 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36009/solr/testSimple1_shard2_replica_n4/
[junit4] 2> 16095 INFO (qtp1774229243-45) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.c.S.Request [testSimple1_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=39
[junit4] 2> 16095 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.u.PeerSync PeerSync: core=testSimple1_shard2_replica_n4 url=http://127.0.0.1:36009/solr START replicas=[http://127.0.0.1:44350/solr/testSimple1_shard2_replica_n5/] nUpdates=100
[junit4] 2> 16127 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 16147 INFO (qtp1856904637-59) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.c.S.Request [testSimple1_shard2_replica_n5] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=12
[junit4] 2> 16154 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:36009_solr) [n:127.0.0.1:36009_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 16154 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:44350_solr) [n:127.0.0.1:44350_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 16350 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.u.PeerSync PeerSync: core=testSimple1_shard1_replica_n3 url=http://127.0.0.1:44350/solr DONE. We have no versions. sync failed.
[junit4] 2> 16351 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 16352 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 16352 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 16362 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44350/solr/testSimple1_shard1_replica_n3/ shard1
[junit4] 2> 16426 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.u.PeerSync PeerSync: core=testSimple1_shard2_replica_n4 url=http://127.0.0.1:36009/solr DONE. We have no versions. sync failed.
[junit4] 2> 16426 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 16426 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 16426 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 16434 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36009/solr/testSimple1_shard2_replica_n4/ shard2
[junit4] 2> 16541 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:44350_solr) [n:127.0.0.1:44350_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 16541 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:36009_solr) [n:127.0.0.1:36009_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 16566 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 16570 INFO (qtp1856904637-35) [n:127.0.0.1:44350_solr c:testSimple1 s:shard1 r:core_node6 x:testSimple1_shard1_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf&newCollection=true&name=testSimple1_shard1_replica_n3&action=CREATE&numShards=2&collection=testSimple1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3467
[junit4] 2> 16587 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 16590 INFO (qtp1774229243-46) [n:127.0.0.1:36009_solr c:testSimple1 s:shard2 r:core_node7 x:testSimple1_shard2_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=testSimple1_shard2_replica_n4&action=CREATE&numShards=2&collection=testSimple1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3482
[junit4] 2> 16694 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:44350_solr) [n:127.0.0.1:44350_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 16694 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:36009_solr) [n:127.0.0.1:36009_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 17040 INFO (qtp1856904637-43) [n:127.0.0.1:44350_solr c:testSimple1 s:shard2 r:core_node8 x:testSimple1_shard2_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=testSimple1_shard2_replica_n5&action=CREATE&numShards=2&collection=testSimple1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3916
[junit4] 2> 17049 INFO (qtp1774229243-44) [n:127.0.0.1:36009_solr c:testSimple1 s:shard1 r:core_node2 x:testSimple1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=testSimple1_shard1_replica_n1&action=CREATE&numShards=2&collection=testSimple1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3947
[junit4] 2> 17064 INFO (qtp1774229243-38) [n:127.0.0.1:36009_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 17128 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 17146 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:36009_solr) [n:127.0.0.1:36009_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 17146 INFO (zkCallback-33-thread-1-processing-n:127.0.0.1:44350_solr) [n:127.0.0.1:44350_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testSimple1/state.json] for collection [testSimple1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 18064 INFO (qtp1774229243-38) [n:127.0.0.1:36009_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&maxShardsPerNode=2&autoAddReplicas=true&name=testSimple1&nrtReplicas=2&action=CREATE&numShards=2&createNodeSet=127.0.0.1:36009_solr,127.0.0.1:44350_solr&wt=javabin&version=2} status=0 QTime=5390
[junit4] 2> 18071 INFO (qtp1774229243-42) [n:127.0.0.1:36009_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=2&autoAddReplicas=false&name=testSimple2&nrtReplicas=2&action=CREATE&numShards=2&createNodeSet=127.0.0.1:44350_solr,127.0.0.1:33278_solr&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 18074 INFO (OverseerCollectionConfigSetProcessor-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 18074 INFO (OverseerThreadFactory-29-thread-2-processing-n:127.0.0.1:33278_solr) [n:127.0.0.1:33278_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection testSimple2
[junit4] 2> 18128 DEBUG (ScheduledTrigger-26-thread-1) [n:127.0.0.1:33278_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 18293 INFO (OverseerStateUpdate-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testSimple2_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33278/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 18307 INFO (OverseerStateUpdate-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple2",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testSimple2_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44350/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 18310 INFO (OverseerStateUpdate-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple2",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testSimple2_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:33278/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 18314 INFO (OverseerStateUpdate-73273076056260615-127.0.0.1:33278_solr-n_0000000000) [n:127.0.0.1:33278_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testSimple2",
[junit4] 2> "sh
[...truncated too long message...]
tricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=251125044
[junit4] 2> 1170287 INFO (coreCloseExecutor-5187-thread-1) [n:127.0.0.1:38855_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=2027244051
[junit4] 2> 1170289 INFO (jetty-closer-6791-thread-5) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@66a48609{/solr,null,UNAVAILABLE}
[junit4] 2> 1170289 INFO (jetty-closer-6791-thread-5) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1170290 INFO (coreCloseExecutor-5188-thread-2) [n:127.0.0.1:36126_solr c:collection1 s:shard1 r:core_node12 x:collection1_shard1_replica_n11] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1.shard1.replica_n11, tag=1059224996
[junit4] 2> 1170290 INFO (coreCloseExecutor-5188-thread-2) [n:127.0.0.1:36126_solr c:collection1 s:shard1 r:core_node12 x:collection1_shard1_replica_n11] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@105f8391: rootName = solr_36126, domain = solr.core.collection1.shard1.replica_n11, service url = null, agent id = null] for registry solr.core.collection1.shard1.replica_n11 / com.codahale.metrics.MetricRegistry@75da3c6b
[junit4] 2> 1170295 INFO (zkCallback-6920-thread-1-processing-n:127.0.0.1:36126_solr) [n:127.0.0.1:36126_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36126_solr
[junit4] 2> 1170296 INFO (jetty-closer-6791-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@49835aa4{/solr,null,UNAVAILABLE}
[junit4] 2> 1170296 INFO (jetty-closer-6791-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1170299 INFO (coreCloseExecutor-5188-thread-2) [n:127.0.0.1:36126_solr c:collection1 s:shard1 r:core_node12 x:collection1_shard1_replica_n11] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=1059224996
[junit4] 2> 1170299 INFO (coreCloseExecutor-5188-thread-1) [n:127.0.0.1:36126_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=466471056
[junit4] 2> 1171661 WARN (zkCallback-6986-thread-1-processing-n:127.0.0.1:44420_solr) [n:127.0.0.1:44420_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1171662 WARN (zkCallback-6963-thread-2-processing-n:127.0.0.1:41970_solr) [n:127.0.0.1:41970_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1171662 INFO (jetty-closer-6791-thread-10) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@762404c1{/solr,null,UNAVAILABLE}
[junit4] 2> 1171662 INFO (jetty-closer-6791-thread-8) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4ba62347{/solr,null,UNAVAILABLE}
[junit4] 2> 1171663 INFO (jetty-closer-6791-thread-10) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1171663 INFO (jetty-closer-6791-thread-8) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1171801 INFO (jetty-closer-6791-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@38f6a0a7{/solr,null,UNAVAILABLE}
[junit4] 2> 1171801 INFO (jetty-closer-6791-thread-4) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1171802 ERROR (SUITE-TriggerIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1171802 INFO (SUITE-TriggerIntegrationTest-seed#[B2B0A5BDC44833E9]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34276 34276
[junit4] 2> 1171967 INFO (Thread-3846) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34276 34276
[junit4] 2> 1171968 WARN (Thread-3846) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 41 /solr/collections/collection1/terms/shard1
[junit4] 2> 19 /solr/clusterprops.json
[junit4] 2> 19 /solr/aliases.json
[junit4] 2> 18 /solr/security.json
[junit4] 2> 11 /solr/configs/conf
[junit4] 2> 9 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 103 /solr/collections/collection1/state.json
[junit4] 2> 42 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 19 /solr/clusterstate.json
[junit4] 2> 6 /solr/autoscaling.json
[junit4] 2> 3 /solr/collections/collection1/leader_elect/shard1/election/73273139632996397-core_node6-n_0000000002
[junit4] 2> 3 /solr/overseer_elect/election/73273139632996375-127.0.0.1:36236_solr-n_0000000006
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73273139632996391-core_node4-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/73273139632996366-127.0.0.1:36255_solr-n_0000000003
[junit4] 2> 2 /solr/overseer_elect/election/73273139632996385-127.0.0.1:32890_solr-n_0000000009
[junit4] 2> 2 /solr/overseer_elect/election/73273139632996403-127.0.0.1:41970_solr-n_0000000015
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 19 /solr/live_nodes
[junit4] 2> 19 /solr/collections
[junit4] 2> 6 /solr/overseer/queue-work
[junit4] 2> 6 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 6 /solr/overseer/collection-queue-work
[junit4] 2> 5 /solr/overseer/queue
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.TriggerIntegrationTest_B2B0A5BDC44833E9-001
[junit4] 2> Feb 10, 2018 8:54:02 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), x_s=PostingsFormat(name=Direct)}, docValues:{}, maxPointsInLeafNode=883, maxMBSortInHeap=6.882601939178405, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@20c1ff37), locale=de-AT, timezone=Europe/Madrid
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=142518912,total=515375104
[junit4] 2> NOTE: All tests run in this JVM: [TestSQLHandler, SaslZkACLProviderTest, NumberUtilsTest, AutoAddReplicasPlanActionTest, TestRangeQuery, TestCorePropertiesReload, TestLazyCores, TestRebalanceLeaders, TestZkAclsWithHadoopAuth, PingRequestHandlerTest, RegexBoostProcessorTest, CachingDirectoryFactoryTest, AddSchemaFieldsUpdateProcessorFactoryTest, MultiTermTest, TestScoreJoinQPScore, TestSortableTextField, CloneFieldUpdateProcessorFactoryTest, PeerSyncReplicationTest, TestCloudSearcherWarming, TestStressLiveNodes, ShufflingReplicaListTransformerTest, TestConfigReload, ClassificationUpdateProcessorFactoryTest, TestHttpShardHandlerFactory, TestRecoveryHdfs, SpellCheckComponentTest, JSONWriterTest, TestDefaultStatsCache, ChaosMonkeyNothingIsSafeWithPullReplicasTest, SegmentsInfoRequestHandlerTest, StressHdfsTest, TestExclusionRuleCollectionAccess, CleanupOldIndexTest, TestTolerantSearch, TestUniqueKeyFieldResource, DirectoryFactoryTest, RestartWhileUpdatingTest, SpatialRPTFieldTypeTest, TestCollectionAPIs, BigEndianAscendingWordSerializerTest, FacetPivotSmallTest, TestSolrQueryResponse, TestSegmentSorting, TestRandomCollapseQParserPlugin, TestQueryTypes, WordBreakSolrSpellCheckerTest, TestManagedSchema, DirectSolrSpellCheckerTest, AnalyticsMergeStrategyTest, SuggesterTSTTest, TestMultiWordSynonyms, TolerantUpdateProcessorTest, CursorMarkTest, HdfsTlogReplayBufferedWhileIndexingTest, TestOnReconnectListenerSupport, TestCodecSupport, DistributedQueryComponentCustomSortTest, TestNumericTerms64, TestReplicationHandlerBackup, ZkShardTermsTest, AnalysisErrorHandlingTest, ZkStateWriterTest, TestAtomicUpdateErrorCases, SuggestComponentTest, TestSuggestSpellingConverter, DateFieldTest, TestSolrDeletionPolicy2, CustomHighlightComponentTest, ForceLeaderTest, TestJmxIntegration, SearchHandlerTest, ResponseBuilderTest, ParsingFieldUpdateProcessorsTest, TestUpdate, ClusterStateTest, ShowFileRequestHandlerTest, TestStressUserVersions, TestSlowCompositeReaderWrapper, TestFieldCacheWithThreads, UtilsToolTest, ExecutePlanActionTest, BaseCdcrDistributedZkTest, TestRealTimeGet, TestDistribStateManager, NoCacheHeaderTest, MoreLikeThisHandlerTest, LeaderElectionIntegrationTest, BasicAuthStandaloneTest, TestBlendedInfixSuggestions, TestSimpleQParserPlugin, TestDistributedMissingSort, TestGraphMLResponseWriter, SolrCloudExampleTest, MetricsHandlerTest, FullHLLTest, NumericFieldsTest, TestPKIAuthenticationPlugin, TestSkipOverseerOperations, TestExactStatsCache, TestSolrConfigHandlerConcurrent, TestExceedMaxTermLength, TestFaceting, TestRawResponseWriter, DocValuesMultiTest, TriggerIntegrationTest]
[junit4] Completed [358/776 (3!)] on J2 in 199.42s, 14 tests, 1 failure <<< FAILURES!
[...truncated 49519 lines...]