You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/02/20 13:08:48 UTC
[JENKINS] Lucene-Solr-Tests-master - Build # 2352 - Failure
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2352/
All tests passed
Build Log:
[...truncated 23998 lines...]
-validate-source-patterns:
[source-patterns] invalid logging pattern [not private static final, uses static class name]: solr/contrib/prometheus-exporter/src/java/org/apache/solr/prometheus/collector/SolrCollector.java
[source-patterns] invalid logging pattern [not private static final, uses static class name]: solr/contrib/prometheus-exporter/src/java/org/apache/solr/prometheus/exporter/SolrExporter.java
[source-patterns] invalid logging pattern [not private static final, uses static class name]: solr/contrib/prometheus-exporter/src/java/org/apache/solr/prometheus/scraper/SolrScraper.java
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:618: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:128: Found 3 violations in source files (invalid logging pattern [not private static final, uses static class name]).
Total time: 81 minutes 1 second
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-master - Build # 2363 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2363/
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardRoutingTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardRoutingTest: 1) Thread[id=9777, name=qtp795916695-9777, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardRoutingTest:
1) Thread[id=9777, name=qtp795916695-9777, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([24B4BD63E0209CAF]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.ShardRoutingTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=9777, name=qtp795916695-9777, state=TIMED_WAITING, group=TGRP-ShardRoutingTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=9777, name=qtp795916695-9777, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([24B4BD63E0209CAF]:0)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue
Error Message:
action wasn't interrupted
Stack Trace:
java.lang.AssertionError: action wasn't interrupted
at __randomizedtesting.SeedInfo.seed([24B4BD63E0209CAF:ED01FFCDE9475A5A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue(TriggerIntegrationTest.java:726)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13112 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> 1054845 INFO (SUITE-TriggerIntegrationTest-seed#[24B4BD63E0209CAF]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_24B4BD63E0209CAF-001/init-core-data-001
[junit4] 2> 1054846 WARN (SUITE-TriggerIntegrationTest-seed#[24B4BD63E0209CAF]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=44 numCloses=44
[junit4] 2> 1054846 INFO (SUITE-TriggerIntegrationTest-seed#[24B4BD63E0209CAF]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1054847 INFO (SUITE-TriggerIntegrationTest-seed#[24B4BD63E0209CAF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1054847 INFO (SUITE-TriggerIntegrationTest-seed#[24B4BD63E0209CAF]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_24B4BD63E0209CAF-001/tempDir-001
[junit4] 2> 1054847 INFO (SUITE-TriggerIntegrationTest-seed#[24B4BD63E0209CAF]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1054848 INFO (Thread-1686) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1054848 INFO (Thread-1686) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1054885 ERROR (Thread-1686) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1054984 INFO (SUITE-TriggerIntegrationTest-seed#[24B4BD63E0209CAF]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:59445
[junit4] 2> 1055007 INFO (zkConnectionManagerCallback-1506-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1055020 INFO (jetty-launcher-1503-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:57:37+05:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1055020 INFO (jetty-launcher-1503-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:57:37+05:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1055049 INFO (jetty-launcher-1503-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1055049 INFO (jetty-launcher-1503-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1055050 INFO (jetty-launcher-1503-thread-2) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1055050 INFO (jetty-launcher-1503-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1055050 INFO (jetty-launcher-1503-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1055050 INFO (jetty-launcher-1503-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1055050 INFO (jetty-launcher-1503-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1d8228ab{/solr,null,AVAILABLE}
[junit4] 2> 1055051 INFO (jetty-launcher-1503-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@16189cb7{HTTP/1.1,[http/1.1]}{127.0.0.1:51188}
[junit4] 2> 1055051 INFO (jetty-launcher-1503-thread-2) [ ] o.e.j.s.Server Started @1057496ms
[junit4] 2> 1055051 INFO (jetty-launcher-1503-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51188}
[junit4] 2> 1055051 ERROR (jetty-launcher-1503-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1055051 INFO (jetty-launcher-1503-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 1055051 INFO (jetty-launcher-1503-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1055051 INFO (jetty-launcher-1503-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1055051 INFO (jetty-launcher-1503-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-23T13:53:23.630Z
[junit4] 2> 1055052 INFO (jetty-launcher-1503-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6824a107{/solr,null,AVAILABLE}
[junit4] 2> 1055053 INFO (jetty-launcher-1503-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@400ad194{HTTP/1.1,[http/1.1]}{127.0.0.1:38443}
[junit4] 2> 1055053 INFO (jetty-launcher-1503-thread-1) [ ] o.e.j.s.Server Started @1057498ms
[junit4] 2> 1055053 INFO (jetty-launcher-1503-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38443}
[junit4] 2> 1055053 ERROR (jetty-launcher-1503-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1055053 INFO (jetty-launcher-1503-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 1055053 INFO (jetty-launcher-1503-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1055053 INFO (jetty-launcher-1503-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1055053 INFO (jetty-launcher-1503-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-23T13:53:23.632Z
[junit4] 2> 1055053 INFO (zkConnectionManagerCallback-1508-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1055054 INFO (jetty-launcher-1503-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1055060 INFO (zkConnectionManagerCallback-1510-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1055061 INFO (jetty-launcher-1503-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1055066 INFO (jetty-launcher-1503-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59445/solr
[junit4] 2> 1055068 INFO (jetty-launcher-1503-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59445/solr
[junit4] 2> 1055097 INFO (zkConnectionManagerCallback-1514-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1055104 INFO (zkConnectionManagerCallback-1518-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1055105 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x10492fcce770004, likely client has closed socket
[junit4] 2> 1055107 INFO (zkConnectionManagerCallback-1520-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1055144 INFO (zkConnectionManagerCallback-1522-thread-1-processing-n:127.0.0.1:38443_solr) [n:127.0.0.1:38443_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1055163 INFO (jetty-launcher-1503-thread-1) [n:127.0.0.1:38443_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1055164 INFO (jetty-launcher-1503-thread-1) [n:127.0.0.1:38443_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:38443_solr
[junit4] 2> 1055165 INFO (jetty-launcher-1503-thread-2) [n:127.0.0.1:51188_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1055165 INFO (jetty-launcher-1503-thread-1) [n:127.0.0.1:38443_solr ] o.a.s.c.Overseer Overseer (id=73345108438089734-127.0.0.1:38443_solr-n_0000000000) starting
[junit4] 2> 1055166 INFO (jetty-launcher-1503-thread-2) [n:127.0.0.1:51188_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51188_solr
[junit4] 2> 1055194 INFO (zkCallback-1521-thread-1-processing-n:127.0.0.1:38443_solr) [n:127.0.0.1:38443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1055194 INFO (zkCallback-1519-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1055218 DEBUG (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1055218 INFO (jetty-launcher-1503-thread-1) [n:127.0.0.1:38443_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38443_solr
[junit4] 2> 1055219 INFO (zkCallback-1519-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1055219 INFO (zkCallback-1521-thread-1-processing-n:127.0.0.1:38443_solr) [n:127.0.0.1:38443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1055233 DEBUG (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1055233 DEBUG (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:51188_solr, 127.0.0.1:38443_solr]
[junit4] 2> 1055233 DEBUG (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1055233 DEBUG (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1055241 DEBUG (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1055241 DEBUG (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1055242 DEBUG (ScheduledTrigger-2576-thread-1) [n:127.0.0.1:38443_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1055277 INFO (jetty-launcher-1503-thread-2) [n:127.0.0.1:51188_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51188.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1055285 INFO (jetty-launcher-1503-thread-2) [n:127.0.0.1:51188_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51188.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1055285 INFO (jetty-launcher-1503-thread-2) [n:127.0.0.1:51188_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51188.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1055286 INFO (jetty-launcher-1503-thread-2) [n:127.0.0.1:51188_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_24B4BD63E0209CAF-001/tempDir-001/node2/.
[junit4] 2> 1055436 INFO (jetty-launcher-1503-thread-1) [n:127.0.0.1:38443_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38443.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1055446 INFO (jetty-launcher-1503-thread-1) [n:127.0.0.1:38443_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38443.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1055447 INFO (jetty-launcher-1503-thread-1) [n:127.0.0.1:38443_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38443.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1055448 INFO (jetty-launcher-1503-thread-1) [n:127.0.0.1:38443_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_24B4BD63E0209CAF-001/tempDir-001/node1/.
[junit4] 2> 1055500 INFO (zkConnectionManagerCallback-1528-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1055525 INFO (zkConnectionManagerCallback-1532-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1055525 INFO (SUITE-TriggerIntegrationTest-seed#[24B4BD63E0209CAF]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1055526 INFO (SUITE-TriggerIntegrationTest-seed#[24B4BD63E0209CAF]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59445/solr ready
[junit4] 2> 1055591 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testContinueTriggersOnOverseerRestart
[junit4] 2> 1055602 INFO (qtp661620047-7989) [n:127.0.0.1:38443_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1055612 INFO (qtp661620047-7989) [n:127.0.0.1:38443_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=10
[junit4] 2> 1055613 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: .auto_add_replicas
[junit4] 2> 1055616 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 1055616 DEBUG (zkCallback-1521-thread-1-processing-n:127.0.0.1:38443_solr) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1055617 DEBUG (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1055617 DEBUG (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1055617 DEBUG (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1055617 DEBUG (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1055617 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1055628 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@400ad194{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1055629 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=431809488
[junit4] 2> 1055629 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1055629 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@614c235a: rootName = solr_38443, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@26112656
[junit4] 2> 1055636 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1055636 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@25e7f6c2: rootName = solr_38443, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4ec2a71
[junit4] 2> 1055640 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1055640 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@457d0b5f: rootName = solr_38443, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6adeaf91
[junit4] 2> 1055642 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:38443_solr
[junit4] 2> 1055642 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1055643 INFO (zkCallback-1519-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1055643 INFO (zkCallback-1521-thread-1-processing-n:127.0.0.1:38443_solr) [n:127.0.0.1:38443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1055645 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1055649 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.Overseer Overseer (id=73345108438089734-127.0.0.1:38443_solr-n_0000000000) closing
[junit4] 2> 1055649 INFO (OverseerStateUpdate-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:38443_solr
[junit4] 2> 1055650 DEBUG (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1055650 WARN (OverseerAutoScalingTriggerThread-73345108438089734-127.0.0.1:38443_solr-n_0000000000) [n:127.0.0.1:38443_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1055651 INFO (zkCallback-1531-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1055651 INFO (zkCallback-1531-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1055652 INFO (zkCallback-1519-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51188_solr
[junit4] 2> 1055652 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6824a107{/solr,null,UNAVAILABLE}
[junit4] 2> 1055652 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1055653 INFO (zkCallback-1519-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.Overseer Overseer (id=73345108438089733-127.0.0.1:51188_solr-n_0000000001) starting
[junit4] 2> 1055665 DEBUG (OverseerAutoScalingTriggerThread-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1055666 DEBUG (OverseerAutoScalingTriggerThread-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1055666 DEBUG (OverseerAutoScalingTriggerThread-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:51188_solr]
[junit4] 2> 1055666 DEBUG (OverseerAutoScalingTriggerThread-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion -1
[junit4] 2> 1055666 DEBUG (OverseerAutoScalingTriggerThread-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1055669 DEBUG (OverseerAutoScalingTriggerThread-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1055669 DEBUG (OverseerAutoScalingTriggerThread-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1055669 DEBUG (ScheduledTrigger-2585-thread-1) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1056673 DEBUG (ScheduledTrigger-2585-thread-1) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1057674 DEBUG (ScheduledTrigger-2585-thread-2) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1058674 DEBUG (ScheduledTrigger-2585-thread-1) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1059674 DEBUG (ScheduledTrigger-2585-thread-1) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1060654 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:57:37+05:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1060655 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1060655 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1060656 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1060656 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3d310d8f{/solr,null,AVAILABLE}
[junit4] 2> 1060657 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5e06b1e3{HTTP/1.1,[http/1.1]}{127.0.0.1:32850}
[junit4] 2> 1060657 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.Server Started @1063102ms
[junit4] 2> 1060657 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=32850}
[junit4] 2> 1060657 ERROR (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1060658 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 1060658 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1060658 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1060658 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-23T13:53:29.237Z
[junit4] 2> 1060662 INFO (zkConnectionManagerCallback-1535-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1060663 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1060673 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59445/solr
[junit4] 2> 1060674 DEBUG (ScheduledTrigger-2585-thread-1) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1060675 INFO (zkConnectionManagerCallback-1539-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1060680 INFO (zkConnectionManagerCallback-1541-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1060691 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:32850_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1060693 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:32850_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1060695 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:32850_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32850_solr
[junit4] 2> 1060697 INFO (zkCallback-1531-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1060697 INFO (zkCallback-1531-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1060697 INFO (zkCallback-1519-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1060705 INFO (zkCallback-1540-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1060781 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:32850_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32850.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1060803 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:32850_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32850.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1060803 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:32850_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32850.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1060804 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:32850_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_24B4BD63E0209CAF-001/tempDir-001/node3/.
[junit4] 2> 1060835 INFO (qtp1380633802-8062) [n:127.0.0.1:32850_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1060845 INFO (qtp1380633802-8062) [n:127.0.0.1:32850_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=10
[junit4] 2> 1060849 INFO (qtp1380633802-8067) [n:127.0.0.1:32850_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1060851 INFO (qtp319484947-8001) [n:127.0.0.1:51188_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 1060851 DEBUG (qtp1380633802-8064) [n:127.0.0.1:32850_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1060855 INFO (qtp1380633802-8064) [n:127.0.0.1:32850_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=8
[junit4] 2> 1060855 DEBUG (zkCallback-1519-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1060856 DEBUG (zkCallback-1519-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:51188_solr, 127.0.0.1:32850_solr]
[junit4] 2> 1060856 DEBUG (zkCallback-1519-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerCTOOR instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 1060856 DEBUG (zkCallback-1519-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:51188_solr, 127.0.0.1:32850_solr]
[junit4] 2> 1060856 DEBUG (OverseerAutoScalingTriggerThread-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1060857 INFO (OverseerAutoScalingTriggerThread-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1060858 DEBUG (ScheduledTrigger-2585-thread-3) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1060858 DEBUG (ScheduledTrigger-2585-thread-3) [n:127.0.0.1:51188_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1060858 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@16189cb7{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1060861 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1760390598
[junit4] 2> 1060861 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1060861 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5a377dec: rootName = solr_51188, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@14b93cb5
[junit4] 2> 1060862 DEBUG (OverseerAutoScalingTriggerThread-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1060869 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1060869 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@59ad1651: rootName = solr_51188, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4ec2a71
[junit4] 2> 1060874 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1060874 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@51d99e28: rootName = solr_51188, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6adeaf91
[junit4] 2> 1060875 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:51188_solr
[junit4] 2> 1060875 INFO (zkCallback-1540-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1060875 INFO (zkCallback-1531-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1060876 INFO (zkCallback-1531-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1060876 INFO (zkCallback-1519-thread-1-processing-n:127.0.0.1:51188_solr) [n:127.0.0.1:51188_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1060876 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1060876 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1060877 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.Overseer Overseer (id=73345108438089733-127.0.0.1:51188_solr-n_0000000001) closing
[junit4] 2> 1060877 INFO (OverseerStateUpdate-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:51188_solr
[junit4] 2> 1060878 WARN (OverseerAutoScalingTriggerThread-73345108438089733-127.0.0.1:51188_solr-n_0000000001) [n:127.0.0.1:51188_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1060879 DEBUG (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1060880 INFO (zkCallback-1540-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:32850_solr
[junit4] 2> 1060880 INFO (zkCallback-1540-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.Overseer Overseer (id=73345108438089739-127.0.0.1:32850_solr-n_0000000002) starting
[junit4] 2> 1060881 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1d8228ab{/solr,null,UNAVAILABLE}
[junit4] 2> 1060881 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1060891 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1060892 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:32850_solr]
[junit4] 2> 1060892 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerCTOOR instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 1060892 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:32850_solr]
[junit4] 2> 1060892 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion -1
[junit4] 2> 1060892 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1060893 INFO (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 1060894 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:51188_solr
[junit4] 2> 1060894 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1060894 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 1060894 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1060895 DEBUG (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1061894 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1061894 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 1061895 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1062895 DEBUG (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1062895 DEBUG (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 1062896 DEBUG (ScheduledTrigger-2596-thread-4) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1063895 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1063895 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 1063896 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1064896 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1064896 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 1064896 DEBUG (ScheduledTrigger-2596-thread-4) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1065896 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1065896 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 1065896 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1066896 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1066896 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 1066896 DEBUG (ScheduledTrigger-2596-thread-4) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1067897 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1067897 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1067897 DEBUG (ScheduledTrigger-2596-thread-1) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 1068897 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1068897 DEBUG (ScheduledTrigger-2596-thread-4) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1068897 DEBUG (ScheduledTrigger-2596-thread-4) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 1069897 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1069897 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1069897 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 1070883 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:57:37+05:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1070885 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1070885 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1070885 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 1070885 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@41521ced{/solr,null,AVAILABLE}
[junit4] 2> 1070886 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@8bb900c{HTTP/1.1,[http/1.1]}{127.0.0.1:41620}
[junit4] 2> 1070886 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.Server Started @1073331ms
[junit4] 2> 1070886 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41620}
[junit4] 2> 1070886 ERROR (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1070887 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 1070887 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1070887 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1070887 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-23T13:53:39.466Z
[junit4] 2> 1070889 INFO (zkConnectionManagerCallback-1545-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1070889 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1070898 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1070898 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1070898 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 1
[junit4] 2> 1070899 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59445/solr
[junit4] 2> 1070900 INFO (zkConnectionManagerCallback-1549-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1070902 INFO (zkConnectionManagerCallback-1551-thread-1-processing-n:127.0.0.1:41620_solr) [n:127.0.0.1:41620_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1070908 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:41620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1070909 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:41620_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1070911 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:41620_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41620_solr
[junit4] 2> 1070917 INFO (zkCallback-1540-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1070918 INFO (zkCallback-1550-thread-1-processing-n:127.0.0.1:41620_solr) [n:127.0.0.1:41620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1070918 INFO (zkCallback-1531-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1070918 INFO (zkCallback-1531-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1071013 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:41620_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41620.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1071023 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:41620_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41620.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1071023 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:41620_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41620.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1071025 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:41620_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_24B4BD63E0209CAF-001/tempDir-001/node4/.
[junit4] 2> 1071898 DEBUG (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1071898 DEBUG (ScheduledTrigger-2596-thread-4) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1071898 DEBUG (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1071898 DEBUG (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:41620_solr at time 19645927697246489
[junit4] 2> 1072899 DEBUG (ScheduledTrigger-2596-thread-4) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1072899 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1072899 DEBUG (ScheduledTrigger-2596-thread-3) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1073899 DEBUG (ScheduledTrigger-2596-thread-4) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1073899 DEBUG (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerCTOOR
[junit4] 2> 1073899 DEBUG (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1073899 DEBUG (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerCTOOR firing registered processor for nodes: [127.0.0.1:41620_solr] added at times [19645927697246489], now=19645929698525470
[junit4] 2> 1073900 DEBUG (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"45cbddfdad6919T4e30cm4g2feiukpwgx3xg9fbm",
[junit4] 2> "source":"node_added_triggerCTOOR",
[junit4] 2> "eventTime":19645927697246489,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[19645927697246489],
[junit4] 2> "nodeNames":["127.0.0.1:41620_solr"]}}
[junit4] 2> 1073900 DEBUG (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_triggerCTOOR]
[junit4] 2> 1073901 INFO (ScheduledTrigger-2596-thread-2) [n:127.0.0.1:32850_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1073902 DEBUG (AutoscalingActionExecutor-2597-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"45cbddfdad6919T4e30cm4g2feiukpwgx3xg9fbm",
[junit4] 2> "source":"node_added_triggerCTOOR",
[junit4] 2> "eventTime":19645927697246489,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[19645927697246489],
[junit4] 2> "_enqueue_time_":19645929699557281,
[junit4] 2> "nodeNames":["127.0.0.1:41620_solr"]}}
[junit4] 2> 1073902 INFO (TEST-TriggerIntegrationTest.testContinueTriggersOnOverseerRestart-seed#[24B4BD63E0209CAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testContinueTriggersOnOverseerRestart
[junit4] 2> 1073907 DEBUG (AutoscalingActionExecutor-2597-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms
[junit4] 2> 1073908 DEBUG (AutoscalingActionExecutor-2597-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerCTOOR after 5000ms
[junit4] 2> 1073912 DEBUG (AutoscalingActionExecutor-2597-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 9 ms for event id=45cbddfdad6919T4e30cm4g2feiukpwgx3xg9fbm
[junit4] 2> 1073921 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSetProperties
[junit4] 2> 1073923 INFO (qtp1380633802-8066) [n:127.0.0.1:32850_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1073928 INFO (qtp1380633802-8066) [n:127.0.0.1:32850_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 1073929 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: .auto_add_replicas
[junit4] 2> 1073930 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: node_added_triggerCTOOR
[junit4] 2> 1073931 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 5
[junit4] 2> 1073931 DEBUG (zkCallback-1540-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1073931 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 1073931 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1073932 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1073932 DEBUG (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 1073932 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1073934 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5e06b1e3{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1073934 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2051132034
[junit4] 2> 1073934 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1073934 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5802c6e2: rootName = solr_32850, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3645edf
[junit4] 2> 1073941 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1073941 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6c6bd4ef: rootName = solr_32850, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@4ec2a71
[junit4] 2> 1073945 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1073945 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1380cab8: rootName = solr_32850, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6adeaf91
[junit4] 2> 1073946 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:32850_solr
[junit4] 2> 1073947 INFO (zkCallback-1550-thread-1-processing-n:127.0.0.1:41620_solr) [n:127.0.0.1:41620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1073948 INFO (zkCallback-1531-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1073949 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1073949 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1073949 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.Overseer Overseer (id=73345108438089739-127.0.0.1:32850_solr-n_0000000002) closing
[junit4] 2> 1073949 WARN (OverseerAutoScalingTriggerThread-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1073949 INFO (OverseerStateUpdate-73345108438089739-127.0.0.1:32850_solr-n_0000000002) [n:127.0.0.1:32850_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:32850_solr
[junit4] 2> 1073950 INFO (zkCallback-1540-thread-1-processing-n:127.0.0.1:32850_solr) [n:127.0.0.1:32850_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1073955 INFO (zkCallback-1531-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1073955 DEBUG (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1073956 INFO (zkCallback-1531-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1073956 INFO (zkCallback-1550-thread-1-processing-n:127.0.0.1:41620_solr) [n:127.0.0.1:41620_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41620_solr
[junit4] 2> 1073957 INFO (zkCallback-1550-thread-1-processing-n:127.0.0.1:41620_solr) [n:127.0.0.1:41620_solr ] o.a.s.c.Overseer Overseer (id=73345108438089742-127.0.0.1:41620_solr-n_0000000003) starting
[junit4] 2> 1073957 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@3d310d8f{/solr,null,UNAVAILABLE}
[junit4] 2> 1073957 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 1073961 DEBUG (OverseerAutoScalingTriggerThread-73345108438089742-127.0.0.1:41620_solr-n_0000000003) [n:127.0.0.1:41620_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1073962 DEBUG (OverseerAutoScalingTriggerThread-73345108438089742-127.0.0.1:41620_solr-n_0000000003) [n:127.0.0.1:41620_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1073962 DEBUG (OverseerAutoScalingTriggerThread-73345108438089742-127.0.0.1:41620_solr-n_0000000003) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:41620_solr]
[junit4] 2> 1073962 DEBUG (OverseerAutoScalingTriggerThread-73345108438089742-127.0.0.1:41620_solr-n_0000000003) [n:127.0.0.1:41620_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
[junit4] 2> 1073962 DEBUG (OverseerAutoScalingTriggerThread-73345108438089742-127.0.0.1:41620_solr-n_0000000003) [n:127.0.0.1:41620_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1073964 DEBUG (OverseerAutoScalingTriggerThread-73345108438089742-127.0.0.1:41620_solr-n_0000000003) [n:127.0.0.1:41620_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1073964 DEBUG (OverseerAutoScalingTriggerThread-73345108438089742-127.0.0.1:41620_solr-n_0000000003) [n:127.0.0.1:41620_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 1073964 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1074965 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1075966 DEBUG (ScheduledTrigger-2607-thread-2) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1076966 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1077967 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1078958 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-22T02:57:37+05:30, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1078962 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1078962 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1078962 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1078963 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@56900ad{/solr,null,AVAILABLE}
[junit4] 2> 1078963 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@e42ebf{HTTP/1.1,[http/1.1]}{127.0.0.1:50159}
[junit4] 2> 1078963 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.e.j.s.Server Started @1081408ms
[junit4] 2> 1078963 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=50159}
[junit4] 2> 1078963 ERROR (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1078964 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 1078964 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1078964 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1078964 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-23T13:53:47.543Z
[junit4] 2> 1078967 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1078975 INFO (zkConnectionManagerCallback-1556-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1078975 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1078994 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59445/solr
[junit4] 2> 1079025 INFO (zkConnectionManagerCallback-1560-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1079033 INFO (zkConnectionManagerCallback-1562-thread-1-processing-n:127.0.0.1:50159_solr) [n:127.0.0.1:50159_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1079039 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:50159_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1079040 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:50159_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1079042 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:50159_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50159_solr
[junit4] 2> 1079045 INFO (zkCallback-1550-thread-1-processing-n:127.0.0.1:41620_solr) [n:127.0.0.1:41620_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1079045 INFO (zkCallback-1531-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1079045 INFO (zkCallback-1531-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1079045 INFO (zkCallback-1531-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1079051 INFO (zkCallback-1561-thread-1-processing-n:127.0.0.1:50159_solr) [n:127.0.0.1:50159_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1079156 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:50159_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50159.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1079166 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:50159_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50159.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1079166 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:50159_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_50159.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7f4734c5
[junit4] 2> 1079167 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [n:127.0.0.1:50159_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_24B4BD63E0209CAF-001/tempDir-001/node5/.
[junit4] 2> 1079967 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1080202 DEBUG (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.ScheduledTriggers Changing value of autoscaling property: triggerScheduleDelaySeconds from: 1 to: 4
[junit4] 2> 1080968 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1081968 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1082968 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1083969 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1084969 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1085969 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1086969 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1087970 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1088205 DEBUG (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.ScheduledTriggers Changing value of autoscaling property: triggerScheduleDelaySeconds from: 4 to: 1
[junit4] 2> 1088970 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1089970 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1090971 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1091971 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1092971 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1093972 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1094972 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1095972 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1096972 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1097973 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1098215 DEBUG (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.ScheduledTriggers Changing value of autoscaling property: triggerCorePoolSize from: 4 to: 6
[junit4] 2> 1098973 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1099973 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1100974 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1101974 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1102975 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1103975 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1104975 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1105975 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1106976 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1107976 DEBUG (ScheduledTrigger-2607-thread-1) [n:127.0.0.1:41620_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1108222 INFO (TEST-TriggerIntegrationTest.testSetProperties-seed#[24B4BD63E0209CAF]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSetProperties
[junit4] 2> 1108235 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[24B4BD63E0209CAF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTrigger
[junit4] 2> 1108236 INFO (qtp730770475-8098) [n:127.0.0.1:41620_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1108242 INFO (qtp730770475-8098) [n:127.0.0.1:41620_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 1108243 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: .auto_add_replicas
[junit4] 2> 1108244 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 7
[junit4] 2> 1108245 DEBUG (zkCallback-1550-thread-1-processing-n:127.0.0.1:41620_solr) [n:127.0.0.1:41620_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1108245 DEBUG (OverseerAutoScalingTriggerThread-73345108438089742-127.0.0.1:41620_solr-n_0000000003) [n:127.0.0.1:41620_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1108245 DEBUG (OverseerAutoScalingTriggerThread-73345108438089742-127.0.0.1:41620_solr-n_0000000003) [n:127.0.0.1:41620_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1108245 DEBUG (OverseerAutoScalingTriggerThread-73345108438089742-127.0.0.1:41620_solr-n_0000000003) [n:127.0.0.1:41620_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1108245 INFO (TEST-TriggerIntegrationTest.testNodeLostTrigger-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1108245 DEBUG (OverseerAutoScalingTriggerThread-73345108438089742-127.0.0.1:41620_solr-n_0000000003) [n:127.0.0.1:41620_
[...truncated too long message...]
[junit4] 2> 1520008 ERROR (TEST-ShardRoutingTest.test-seed#[24B4BD63E0209CAF]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1520008 INFO (TEST-ShardRoutingTest.test-seed#[24B4BD63E0209CAF]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42625 42625
[junit4] 2> 1520185 INFO (Thread-2158) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42625 42625
[junit4] 2> 1520186 WARN (Thread-2158) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/aliases.json
[junit4] 2> 9 /solr/security.json
[junit4] 2> 9 /solr/configs/conf1
[junit4] 2> 5 /solr/collections/collection1/terms/shard1
[junit4] 2> 5 /solr/collections/collection1/terms/shard2
[junit4] 2> 5 /solr/collections/collection1/terms/shard3
[junit4] 2> 5 /solr/collections/collection1/terms/shard4
[junit4] 2> 2 /solr/collections/control_collection/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/clusterprops.json
[junit4] 2> 11 /solr/clusterstate.json
[junit4] 2> 9 /solr/autoscaling.json
[junit4] 2> 8 /solr/collections/collection1/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 11 /solr/live_nodes
[junit4] 2> 11 /solr/collections
[junit4] 2> 9 /solr/overseer/queue
[junit4] 2> 9 /solr/overseer/queue-work
[junit4] 2> 9 /solr/overseer/collection-queue-work
[junit4] 2> 8 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2>
[junit4] 2> 1520187 INFO (SUITE-ShardRoutingTest-seed#[24B4BD63E0209CAF]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 23, 2018 2:01:09 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 23, 2018 2:01:29 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardRoutingTest:
[junit4] 2> 1) Thread[id=9777, name=qtp795916695-9777, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 23, 2018 2:01:29 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=9777, name=qtp795916695-9777, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
[junit4] 2> Feb 23, 2018 2:01:32 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=9777, name=qtp795916695-9777, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=Memory)}, docValues:{foo_i=DocValuesFormat(name=Lucene70), _version_=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1275, maxMBSortInHeap=5.19073530171366, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@70192e62), locale=ar-SA, timezone=America/Pangnirtung
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=195209176,total=520093696
[junit4] 2> NOTE: All tests run in this JVM: [PluginInfoTest, TestPHPSerializedResponseWriter, BinaryUpdateRequestHandlerTest, TestPullReplicaErrorHandling, TestDistribStateManager, TestRebalanceLeaders, TestClusterProperties, TestCrossCoreJoin, TestHdfsBackupRestoreCore, TestCollectionsAPIViaSolrCloudCluster, TestFastOutputStream, OverseerCollectionConfigSetProcessorTest, TestHalfAndHalfDocValues, TestFreeTextSuggestions, DistributedTermsComponentTest, ShardSplitTest, TestUseDocValuesAsStored2, TestManagedSchemaThreadSafety, SuggestComponentContextFilterQueryTest, CdcrReplicationDistributedZkTest, TestConfig, TestTrackingShardHandlerFactory, OutputWriterTest, TestGroupingSearch, ChaosMonkeyNothingIsSafeTest, SecurityConfHandlerTest, TestCollationField, TestSegmentSorting, TestEmbeddedSolrServerSchemaAPI, TestSkipOverseerOperations, TestRestoreCore, TestMultiValuedNumericRangeQuery, TestManagedSynonymFilterFactory, ReplicaListTransformerTest, TestStandardQParsers, BlockDirectoryTest, DistribDocExpirationUpdateProcessorTest, DirectoryFactoryTest, TestInitParams, BaseCdcrDistributedZkTest, TestLargeCluster, TestSolrXml, TestStressLucene, TestMergePolicyConfig, TestDynamicLoading, TestRemoteStreaming, LeaderFailureAfterFreshStartTest, StressHdfsTest, TestQueryWrapperFilter, SharedFSAutoReplicaFailoverTest, TestTestInjection, DateRangeFieldTest, DeleteLastCustomShardedReplicaTest, TestOmitPositions, TestSchemalessBufferedUpdates, SimpleFacetsTest, TestDocumentBuilder, TestOrdValues, TriggerIntegrationTest, SolrSlf4jReporterTest, QueryResultKeyTest, TestDistributedSearch, HLLUtilTest, ZkShardTermsTest, TestLegacyFieldReuse, TestPseudoReturnFields, TestPhraseSuggestions, RulesTest, UUIDFieldTest, ActionThrottleTest, TestRandomFaceting, ShardRoutingTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=ShardRoutingTest -Dtests.seed=24B4BD63E0209CAF -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=ar-SA -Dtests.timezone=America/Pangnirtung -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | ShardRoutingTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.ShardRoutingTest:
[junit4] > 1) Thread[id=9777, name=qtp795916695-9777, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([24B4BD63E0209CAF]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=9777, name=qtp795916695-9777, state=TIMED_WAITING, group=TGRP-ShardRoutingTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([24B4BD63E0209CAF]:0)
[junit4] Completed [465/778 (2!)] on J0 in 76.08s, 1 test, 2 errors <<< FAILURES!
[...truncated 51064 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2362 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2362/
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([3ED79B0344BF6E99:639F858A8B79C8D6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([3ED79B0344BF6E99:532B3FFEFEF7919E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:313)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12729 lines...]
[junit4] Suite: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest
[junit4] 2> 1457323 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[3ED79B0344BF6E99]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/init-core-data-001
[junit4] 2> 1457341 WARN (SUITE-AutoscalingHistoryHandlerTest-seed#[3ED79B0344BF6E99]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 1457342 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[3ED79B0344BF6E99]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1457343 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[3ED79B0344BF6E99]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 1457344 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[3ED79B0344BF6E99]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001
[junit4] 2> 1457344 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[3ED79B0344BF6E99]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1457382 INFO (Thread-987) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1457382 INFO (Thread-987) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1457418 ERROR (Thread-987) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1457487 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[3ED79B0344BF6E99]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:35096
[junit4] 2> 1457517 INFO (zkConnectionManagerCallback-857-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1457531 INFO (jetty-launcher-854-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T11:27:37-10:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1457548 INFO (jetty-launcher-854-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T11:27:37-10:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 1457560 INFO (jetty-launcher-854-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1457560 INFO (jetty-launcher-854-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1457560 INFO (jetty-launcher-854-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1457560 INFO (jetty-launcher-854-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@38c86ea6{/solr,null,AVAILABLE}
[junit4] 2> 1457561 INFO (jetty-launcher-854-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4bc9ef7a{HTTP/1.1,[http/1.1]}{127.0.0.1:44022}
[junit4] 2> 1457561 INFO (jetty-launcher-854-thread-1) [ ] o.e.j.s.Server Started @1467901ms
[junit4] 2> 1457561 INFO (jetty-launcher-854-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44022}
[junit4] 2> 1457562 ERROR (jetty-launcher-854-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1457562 INFO (jetty-launcher-854-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 1457562 INFO (jetty-launcher-854-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1457562 INFO (jetty-launcher-854-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1457562 INFO (jetty-launcher-854-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-23T08:29:09.561Z
[junit4] 2> 1457577 INFO (jetty-launcher-854-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 1457577 INFO (jetty-launcher-854-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 1457578 INFO (jetty-launcher-854-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 1457578 INFO (jetty-launcher-854-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@469083e1{/solr,null,AVAILABLE}
[junit4] 2> 1457578 INFO (jetty-launcher-854-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@12681a01{HTTP/1.1,[http/1.1]}{127.0.0.1:36426}
[junit4] 2> 1457578 INFO (jetty-launcher-854-thread-2) [ ] o.e.j.s.Server Started @1467918ms
[junit4] 2> 1457578 INFO (jetty-launcher-854-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36426}
[junit4] 2> 1457578 ERROR (jetty-launcher-854-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1457578 INFO (jetty-launcher-854-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 1457579 INFO (jetty-launcher-854-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1457579 INFO (jetty-launcher-854-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1457579 INFO (jetty-launcher-854-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-23T08:29:09.578Z
[junit4] 2> 1457645 INFO (zkConnectionManagerCallback-859-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1457645 INFO (zkConnectionManagerCallback-861-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1457670 INFO (jetty-launcher-854-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1457670 INFO (jetty-launcher-854-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1457702 INFO (jetty-launcher-854-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35096/solr
[junit4] 2> 1457737 INFO (jetty-launcher-854-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35096/solr
[junit4] 2> 1457751 INFO (zkConnectionManagerCallback-865-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1457753 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x100660a49ef0003, likely client has closed socket
[junit4] 2> 1457767 INFO (zkConnectionManagerCallback-869-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1457787 INFO (zkConnectionManagerCallback-871-thread-1-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1457817 INFO (zkConnectionManagerCallback-873-thread-1-processing-n:127.0.0.1:44022_solr) [n:127.0.0.1:44022_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1457975 INFO (jetty-launcher-854-thread-2) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1457975 INFO (jetty-launcher-854-thread-2) [n:127.0.0.1:36426_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:36426_solr
[junit4] 2> 1457976 INFO (jetty-launcher-854-thread-2) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer Overseer (id=72169788414033925-127.0.0.1:36426_solr-n_0000000000) starting
[junit4] 2> 1458144 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 1458145 INFO (jetty-launcher-854-thread-2) [n:127.0.0.1:36426_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36426_solr
[junit4] 2> 1458146 DEBUG (OverseerAutoScalingTriggerThread-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1458147 INFO (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1458193 DEBUG (OverseerAutoScalingTriggerThread-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1458193 DEBUG (OverseerAutoScalingTriggerThread-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:36426_solr]
[junit4] 2> 1458193 DEBUG (OverseerAutoScalingTriggerThread-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 1458193 DEBUG (OverseerAutoScalingTriggerThread-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1458198 DEBUG (OverseerAutoScalingTriggerThread-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1458198 DEBUG (OverseerAutoScalingTriggerThread-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1458199 DEBUG (ScheduledTrigger-1367-thread-1) [n:127.0.0.1:36426_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 1458447 INFO (jetty-launcher-854-thread-1) [n:127.0.0.1:44022_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1458466 INFO (jetty-launcher-854-thread-1) [n:127.0.0.1:44022_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1458468 INFO (jetty-launcher-854-thread-1) [n:127.0.0.1:44022_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44022_solr
[junit4] 2> 1458514 INFO (zkCallback-870-thread-1-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1458515 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:44022_solr"} current state version: 0
[junit4] 2> 1458515 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:44022_solr
[junit4] 2> 1458535 INFO (zkCallback-872-thread-1-processing-n:127.0.0.1:44022_solr) [n:127.0.0.1:44022_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1458694 INFO (jetty-launcher-854-thread-2) [n:127.0.0.1:36426_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36426.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7fe25c83
[junit4] 2> 1458764 INFO (jetty-launcher-854-thread-2) [n:127.0.0.1:36426_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36426.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7fe25c83
[junit4] 2> 1458764 INFO (jetty-launcher-854-thread-2) [n:127.0.0.1:36426_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36426.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7fe25c83
[junit4] 2> 1458778 INFO (jetty-launcher-854-thread-2) [n:127.0.0.1:36426_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node2/.
[junit4] 2> 1458934 INFO (jetty-launcher-854-thread-1) [n:127.0.0.1:44022_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44022.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7fe25c83
[junit4] 2> 1458970 INFO (jetty-launcher-854-thread-1) [n:127.0.0.1:44022_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44022.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7fe25c83
[junit4] 2> 1458970 INFO (jetty-launcher-854-thread-1) [n:127.0.0.1:44022_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44022.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7fe25c83
[junit4] 2> 1458983 INFO (jetty-launcher-854-thread-1) [n:127.0.0.1:44022_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/.
[junit4] 2> 1459049 INFO (zkConnectionManagerCallback-879-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1459085 INFO (zkConnectionManagerCallback-883-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 1459087 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[3ED79B0344BF6E99]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1459087 INFO (SUITE-AutoscalingHistoryHandlerTest-seed#[3ED79B0344BF6E99]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35096/solr ready
[junit4] 2> 1459109 INFO (qtp913925118-4066) [n:127.0.0.1:44022_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&name=.system&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:36426_solr&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1459162 INFO (OverseerThreadFactory-1369-thread-1-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection .system
[junit4] 2> 1459176 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":".system",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "numShards":1,
[junit4] 2> "createNodeSet":"127.0.0.1:36426_solr",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 1459176 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ClusterStateMutator building a new cName: .system
[junit4] 2> 1459176 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/.system/state.json
[junit4] 2> 1459206 DEBUG (ScheduledTrigger-1367-thread-1) [n:127.0.0.1:36426_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1459341 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36426/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 1459341 INFO (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36426/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1459342 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 1459342 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node2":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:36426/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 1459479 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/.system/state.json version: 0
[junit4] 2> 1459608 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n1&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1459608 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1459702 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:36426/solr",
[junit4] 2> "node_name":"127.0.0.1:36426_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":".system",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1459702 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:36426/solr",
[junit4] 2> "node_name":"127.0.0.1:36426_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":".system",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 1459702 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node2:{"core":".system_shard1_replica_n1","base_url":"http://127.0.0.1:36426/solr","node_name":"127.0.0.1:36426_solr","state":"down","type":"NRT"}
[junit4] 2> 1459702 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(.system//collections/.system/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node2":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:36426/solr",
[junit4] 2> "node_name":"127.0.0.1:36426_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 1459815 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/.system/state.json version: 1
[junit4] 2> 1459817 INFO (zkCallback-870-thread-1-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1459817 INFO (zkCallback-870-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1460212 DEBUG (ScheduledTrigger-1367-thread-2) [n:127.0.0.1:36426_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1460760 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 1460781 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 1460796 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.s.IndexSchema [.system_shard1_replica_n1] Schema name=_system collection or core
[junit4] 2> 1460817 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 1460831 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/.system/managed-schema
[junit4] 2> 1460833 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/.system/schema.xml to /configs/.system/schema.xml.bak
[junit4] 2> 1460833 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_n1' using configuration from collection .system, trusted=true
[junit4] 2> 1460834 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36426.solr.core..system.shard1.replica_n1' (registry 'solr.core..system.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7fe25c83
[junit4] 2> 1460834 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1460834 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [[.system_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node2/.system_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node2/./.system_shard1_replica_n1/data/]
[junit4] 2> 1461180 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1461180 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1461181 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if 1 uncommited docs
[junit4] 2> 1461181 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1461199 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1f860d21[.system_shard1_replica_n1] main]
[junit4] 2> 1461213 DEBUG (ScheduledTrigger-1367-thread-1) [n:127.0.0.1:36426_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1461220 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/.system
[junit4] 2> 1461220 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/.system
[junit4] 2> 1461221 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/.system/managed-schema
[junit4] 2> 1461221 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1461222 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1461223 INFO (searcherExecutor-1374-thread-1-processing-n:127.0.0.1:36426_solr x:.system_shard1_replica_n1 s:shard1 c:.system r:core_node2) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [.system_shard1_replica_n1] Registered new searcher Searcher@1f860d21[.system_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1461223 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593179691519311872
[junit4] 2> 1461237 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1461237 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1461237 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36426/solr/.system_shard1_replica_n1/
[junit4] 2> 1461237 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1461237 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:36426/solr/.system_shard1_replica_n1/ has no replicas
[junit4] 2> 1461237 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1461253 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36426/solr/.system_shard1_replica_n1/ shard1
[junit4] 2> 1461253 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":".system",
[junit4] 2> "base_url":"http://127.0.0.1:36426/solr",
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "state":"active"} current state version: 0
[junit4] 2> 1461354 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/.system/state.json version: 2
[junit4] 2> 1461354 INFO (zkCallback-870-thread-3-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1461354 INFO (zkCallback-870-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1461404 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1461405 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:36426/solr",
[junit4] 2> "node_name":"127.0.0.1:36426_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":".system",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1461405 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node2",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:36426/solr",
[junit4] 2> "node_name":"127.0.0.1:36426_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"active",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":".system",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 1461405 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node2:{"core":".system_shard1_replica_n1","base_url":"http://127.0.0.1:36426/solr","node_name":"127.0.0.1:36426_solr","state":"active","type":"NRT","leader":"true"}
[junit4] 2> 1461405 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(.system//collections/.system/state.json/3)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"1",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node2":{
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:36426/solr",
[junit4] 2> "node_name":"127.0.0.1:36426_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "leader":"true"}}}}}
[junit4] 2> 1461405 INFO (qtp747239503-4077) [n:127.0.0.1:36426_solr c:.system s:shard1 r:core_node2 x:.system_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n1&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1797
[junit4] 2> 1461408 INFO (qtp913925118-4066) [n:127.0.0.1:44022_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1461506 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/.system/state.json version: 3
[junit4] 2> 1461506 INFO (zkCallback-870-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1461506 INFO (zkCallback-870-thread-3-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1462213 DEBUG (ScheduledTrigger-1367-thread-1) [n:127.0.0.1:36426_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1462409 INFO (qtp913925118-4066) [n:127.0.0.1:44022_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&name=.system&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:36426_solr&wt=javabin&version=2} status=0 QTime=3299
[junit4] 2> 1462410 INFO (qtp913925118-4068) [n:127.0.0.1:44022_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=3&maxShardsPerNode=3&name=AutoscalingHistoryHandlerTest_collection&nrtReplicas=3&action=CREATE&numShards=1&createNodeSet=127.0.0.1:44022_solr&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1462412 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection AutoscalingHistoryHandlerTest_collection
[junit4] 2> 1462413 INFO (OverseerCollectionConfigSetProcessor-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1462413 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/protwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/protwords.txt
[junit4] 2> 1462414 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/managed-schema to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/managed-schema
[junit4] 2> 1462416 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/solrconfig.xml to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/solrconfig.xml
[junit4] 2> 1462417 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/synonyms.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/synonyms.txt
[junit4] 2> 1462418 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/stopwords.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/stopwords.txt
[junit4] 2> 1462420 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/params.json to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/params.json
[junit4] 2> 1462421 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_it.txt
[junit4] 2> 1462422 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_no.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_no.txt
[junit4] 2> 1462424 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hy.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hy.txt
[junit4] 2> 1462433 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stemdict_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stemdict_nl.txt
[junit4] 2> 1462434 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ca.txt
[junit4] 2> 1462435 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_id.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_id.txt
[junit4] 2> 1462437 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ga.txt
[junit4] 2> 1462438 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_da.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_da.txt
[junit4] 2> 1462439 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hi.txt
[junit4] 2> 1462440 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ja.txt
[junit4] 2> 1462442 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_pt.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_pt.txt
[junit4] 2> 1462443 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_hu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_hu.txt
[junit4] 2> 1462444 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_el.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_el.txt
[junit4] 2> 1462445 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ru.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ru.txt
[junit4] 2> 1462447 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_tr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_tr.txt
[junit4] 2> 1462448 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ar.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ar.txt
[junit4] 2> 1462457 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/userdict_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/userdict_ja.txt
[junit4] 2> 1462458 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_it.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_it.txt
[junit4] 2> 1462460 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_eu.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_eu.txt
[junit4] 2> 1462461 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_cz.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_cz.txt
[junit4] 2> 1462462 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stoptags_ja.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stoptags_ja.txt
[junit4] 2> 1462509 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_fr.txt
[junit4] 2> 1462510 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fa.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fa.txt
[junit4] 2> 1462512 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_de.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_de.txt
[junit4] 2> 1462513 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/contractions_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/contractions_ga.txt
[junit4] 2> 1462527 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ca.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ca.txt
[junit4] 2> 1462528 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_nl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_nl.txt
[junit4] 2> 1462529 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_bg.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_bg.txt
[junit4] 2> 1462534 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_es.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_es.txt
[junit4] 2> 1462536 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_en.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_en.txt
[junit4] 2> 1462538 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_lv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_lv.txt
[junit4] 2> 1462547 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_th.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_th.txt
[junit4] 2> 1462548 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_ro.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_ro.txt
[junit4] 2> 1462550 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fr.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fr.txt
[junit4] 2> 1462551 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_sv.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_sv.txt
[junit4] 2> 1462552 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_fi.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_fi.txt
[junit4] 2> 1462554 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/hyphenations_ga.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/hyphenations_ga.txt
[junit4] 2> 1462555 INFO (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.c.ZkConfigManager Copying zk node /configs/_default/lang/stopwords_gl.txt to /configs/AutoscalingHistoryHandlerTest_collection.AUTOCREATED/lang/stopwords_gl.txt
[junit4] 2> 1462556 WARN (OverseerThreadFactory-1369-thread-2-processing-n:127.0.0.1:36426_solr) [n:127.0.0.1:36426_solr ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 3 on collection AutoscalingHistoryHandlerTest_collection is higher than the number of Solr instances currently live or live and part of your createNodeSet(1). It's unusual to run two replica of the same slice on the same Solr-instance.
[junit4] 2> 1462561 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "numShards":"1",
[junit4] 2> "maxShardsPerNode":"3",
[junit4] 2> "createNodeSet":"127.0.0.1:44022_solr",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 1462561 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ClusterStateMutator building a new cName: AutoscalingHistoryHandlerTest_collection
[junit4] 2> 1462661 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json
[junit4] 2> 1462765 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 1462765 INFO (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1462766 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 1462766 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 1462767 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 1462767 INFO (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1462768 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 1462768 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 1462795 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 1462795 INFO (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 1462797 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 1462797 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 1462898 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 0
[junit4] 2> 1462998 INFO (qtp913925118-4071) [n:127.0.0.1:44022_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1462999 INFO (qtp913925118-4065) [n:127.0.0.1:44022_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n4&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1462999 INFO (qtp913925118-4071) [n:127.0.0.1:44022_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1462999 INFO (qtp913925118-4141) [n:127.0.0.1:44022_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=AutoscalingHistoryHandlerTest_collection.AUTOCREATED&newCollection=true&name=AutoscalingHistoryHandlerTest_collection_shard1_replica_n2&action=CREATE&numShards=1&collection=AutoscalingHistoryHandlerTest_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1463004 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1463004 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 1463004 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1","base_url":"http://127.0.0.1:44022/solr","node_name":"127.0.0.1:44022_solr","state":"down","type":"NRT"}
[junit4] 2> 1463004 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"3",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 1463005 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1463005 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "core_node_name":"core_node5",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 1463005 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node5:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2","base_url":"http://127.0.0.1:44022/solr","node_name":"127.0.0.1:44022_solr","state":"down","type":"NRT"}
[junit4] 2> 1463005 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"3",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 1463013 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 1463013 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=1 message={
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "core_node_name":"core_node6",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "numShards":"1",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"AutoscalingHistoryHandlerTest_collection",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 1463013 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node6:{"core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4","base_url":"http://127.0.0.1:44022/solr","node_name":"127.0.0.1:44022_solr","state":"down","type":"NRT"}
[junit4] 2> 1463013 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(AutoscalingHistoryHandlerTest_collection//collections/AutoscalingHistoryHandlerTest_collection/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"3",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"3",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"3",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{"shard1":{
[junit4] 2> "range":"80000000-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{
[junit4] 2> "core_node3":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node5":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"},
[junit4] 2> "core_node6":{
[junit4] 2> "core":"AutoscalingHistoryHandlerTest_collection_shard1_replica_n4",
[junit4] 2> "base_url":"http://127.0.0.1:44022/solr",
[junit4] 2> "node_name":"127.0.0.1:44022_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 1463114 DEBUG (OverseerStateUpdate-72169788414033925-127.0.0.1:36426_solr-n_0000000000) [n:127.0.0.1:36426_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/AutoscalingHistoryHandlerTest_collection/state.json version: 1
[junit4] 2> 1463115 INFO (zkCallback-872-thread-1-processing-n:127.0.0.1:44022_solr) [n:127.0.0.1:44022_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1463115 INFO (zkCallback-872-thread-2-processing-n:127.0.0.1:44022_solr) [n:127.0.0.1:44022_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/AutoscalingHistoryHandlerTest_collection/state.json] for collection [AutoscalingHistoryHandlerTest_collection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1463213 DEBUG (ScheduledTrigger-1367-thread-1) [n:127.0.0.1:36426_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1464091 INFO (qtp913925118-4141) [n:127.0.0.1:44022_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 1464091 INFO (qtp913925118-4141) [n:127.0.0.1:44022_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 1464092 WARN (qtp913925118-4141) [n:127.0.0.1:44022_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../contrib/extraction/lib
[junit4] 2> 1464092 WARN (qtp913925118-4141) [n:127.0.0.1:44022_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node5 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n2] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n2/../../../../dist
[junit4] 2> 1464108 INFO (qtp913925118-4071) [n:127.0.0.1:44022_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams conf resource params.json loaded . version : 0
[junit4] 2> 1464125 INFO (qtp913925118-4071) [n:127.0.0.1:44022_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.RequestParams request params refreshed to version 0
[junit4] 2> 1464126 WARN (qtp913925118-4071) [n:127.0.0.1:44022_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/extraction/lib
[junit4] 2> 1464126 WARN (qtp913925118-4071) [n:127.0.0.1:44022_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-cell-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 1464126 WARN (qtp913925118-4071) [n:127.0.0.1:44022_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib filtered by .*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../contrib/clustering/lib
[junit4] 2> 1464126 WARN (qtp913925118-4071) [n:127.0.0.1:44022_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist filtered by solr-clustering-\d.*\.jar to classpath: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.admin.AutoscalingHistoryHandlerTest_3ED79B0344BF6E99-001/tempDir-001/node1/AutoscalingHistoryHandlerTest_collection_shard1_replica_n1/../../../../dist
[junit4] 2> 1464126 WARN (qtp913925118-4071) [n:127.0.0.1:44022_solr c:AutoscalingHistoryHandlerTest_collection s:shard1 r:core_node3 x:AutoscalingHistoryHandlerTest_collection_shard1_replica_n1] o.a.s.c.SolrConfig Couldn't add files from /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.handler.a
[...truncated too long message...]
38-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"628a1d3435ca3T5c0i5po49rkbm5n07q4g7po04",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":1733525359516835,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1733525359516835],
[junit4] 2> "nodeNames":["127.0.0.1:10022_solr"]}}
[junit4] 2> 2807474 DEBUG (ScheduledTrigger-4438-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 2807485 DEBUG (AutoscalingActionExecutor-4439-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"628a1d3435ca3T5c0i5po49rkbm5n07q4g7po04",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":1733525359516835,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1733525359516835],
[junit4] 2> "_enqueue_time_":1733530263900585,
[junit4] 2> "nodeNames":["127.0.0.1:10022_solr"]}}
[junit4] 2> 2807506 DEBUG (AutoscalingActionExecutor-4439-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 2807506 DEBUG (AutoscalingActionExecutor-4439-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 2807506 DEBUG (AutoscalingActionExecutor-4439-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 1026 ms for event id=628a1d3435ca3T5c0i5po49rkbm5n07q4g7po04
[junit4] 2> 2807506 DEBUG (simCloudManagerPool-4437-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2807506 DEBUG (simCloudManagerPool-4437-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 45
[junit4] 2> 2807506 DEBUG (simCloudManagerPool-4437-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10022_solr, 127.0.0.1:10021_solr, 127.0.0.1:10019_solr]
[junit4] 2> 2807506 DEBUG (simCloudManagerPool-4437-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10022_solr, 127.0.0.1:10021_solr, 127.0.0.1:10019_solr]
[junit4] 2> 2807506 DEBUG (simCloudManagerPool-4437-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 2807506 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 45
[junit4] 2> 2807507 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 45, lastZnodeVersion 45
[junit4] 2> 2807554 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTrigger
[junit4] 2> 2807554 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 2807554 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 2807554 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 2807554 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 2807554 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 2807554 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 2807554 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 2807554 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 2807554 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 2807554 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 2807555 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 2807555 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 2807555 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 2807555 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 2807555 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 2807555 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger
[junit4] 2> 2807555 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 2807555 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 2807555 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: search_rate_trigger
[junit4] 2> 2807555 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[3ED79B0344BF6E99]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - FAILED 1
[junit4] 2> 2807555 DEBUG (SUITE-TestTriggerIntegration-seed#[3ED79B0344BF6E99]-worker) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_3ED79B0344BF6E99-001
[junit4] 2> 2807557 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@3a5af99a), locale=zh, timezone=NET
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=202645352,total=529530880
[junit4] 2> NOTE: All tests run in this JVM: [AutoAddReplicasIntegrationTest, HdfsDirectoryTest, DeleteInactiveReplicaTest, TestSolrCloudWithKerberosAlt, UUIDUpdateProcessorFallbackTest, CollectionsAPIDistributedZkTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestDirectoryFactory, TestJavabinTupleStreamParser, CacheHeaderTest, SaslZkACLProviderTest, HdfsChaosMonkeySafeLeaderTest, TestCollectionAPI, TestLegacyTerms, TestStressInPlaceUpdates, TestSchemaNameResource, TestPolicyCloud, TestNamedUpdateProcessors, TestRandomDVFaceting, SchemaApiFailureTest, ClusterStateTest, MultiTermTest, DistributedFacetSimpleRefinementLongTailTest, TestDynamicFieldCollectionResource, TestReloadDeadlock, CoreMergeIndexesAdminHandlerTest, TestJmxIntegration, TestDeleteCollectionOnDownNodes, ShowFileRequestHandlerTest, TestSchemaVersionResource, AutoscalingHistoryHandlerTest, MultiThreadedOCPTest, PreAnalyzedFieldTest, TestTolerantUpdateProcessorCloud, DistributedQueueTest, TestSolrCLIRunExample, CollectionReloadTest, TestFiltering, TestSolrCoreProperties, SolrCloudExampleTest, SpellCheckComponentTest, URLClassifyProcessorTest, MoveReplicaHDFSFailoverTest, EnumFieldTest, HdfsCollectionsAPIDistributedZkTest, JvmMetricsTest, DeleteReplicaTest, LeaderFailoverAfterPartitionTest, TestSolrFieldCacheBean, LeaderElectionIntegrationTest, DistributedQueryComponentOptimizationTest, DistributedFacetPivotLargeTest, TestDownShardTolerantSearch, LargeFieldTest, TestUtils, TestCloudSearcherWarming, SearchHandlerTest, TlogReplayBufferedWhileIndexingTest, DistributedSpellCheckComponentTest, TestExportWriter, TestNoOpRegenerator, InfixSuggestersTest, BadCopyFieldTest, TestFieldCache, TestConfigOverlay, TestCollationFieldDocValues, ConvertedLegacyTest, UnloadDistributedZkTest, TestRestManager, TestRuleBasedAuthorizationPlugin, BlockJoinFacetSimpleTest, TestComputePlanAction, DocValuesMultiTest, LegacyCloudClusterPropTest, ClusterStateUpdateTest, CollectionsAPISolrJTest, TestNestedDocsSort, TestMiniSolrCloudClusterSSL, SolrCmdDistributorTest, TestRealTimeGet, LeaderElectionTest, SpellCheckCollatorWithCollapseTest, SolrShardReporterTest, CircularListTest, PeerSyncReplicationTest, TestLRUStatsCache, TestCodecSupport, SpatialFilterTest, ResponseLogComponentTest, AtomicUpdateProcessorFactoryTest, TestTriggerIntegration]
[junit4] Completed [360/778 (2!)] on J2 in 7.09s, 11 tests, 1 failure <<< FAILURES!
[...truncated 51466 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2361 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2361/
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.io.stream.SelectWithEvaluatorsTest
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.client.solrj.io.stream.SelectWithEvaluatorsTest: 1) Thread[id=134, name=qtp1320716203-134, state=TIMED_WAITING, group=TGRP-SelectWithEvaluatorsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.io.stream.SelectWithEvaluatorsTest:
1) Thread[id=134, name=qtp1320716203-134, state=TIMED_WAITING, group=TGRP-SelectWithEvaluatorsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([32AE9B573185ED25]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.io.stream.SelectWithEvaluatorsTest
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=134, name=qtp1320716203-134, state=TIMED_WAITING, group=TGRP-SelectWithEvaluatorsTest] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=134, name=qtp1320716203-134, state=TIMED_WAITING, group=TGRP-SelectWithEvaluatorsTest]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([32AE9B573185ED25]:0)
Build Log:
[...truncated 15380 lines...]
[junit4] Suite: org.apache.solr.client.solrj.io.stream.SelectWithEvaluatorsTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.SelectWithEvaluatorsTest_32AE9B573185ED25-001/init-core-data-001
[junit4] 2> 11111 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 11112 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 11112 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 11115 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.SelectWithEvaluatorsTest_32AE9B573185ED25-001/tempDir-001
[junit4] 2> 11143 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 11162 INFO (Thread-31) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 11162 INFO (Thread-31) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 11260 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:58591
[junit4] 2> 11324 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Exception causing close of session 0x0: ZooKeeperServer not running
[junit4] 2> 11425 ERROR (Thread-31) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 13024 INFO (zkConnectionManagerCallback-10-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13075 INFO (jetty-launcher-7-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 13075 INFO (jetty-launcher-7-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 13077 INFO (jetty-launcher-7-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 13081 INFO (jetty-launcher-7-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 13082 INFO (jetty-launcher-7-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 13082 INFO (jetty-launcher-7-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 13082 INFO (jetty-launcher-7-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 13082 INFO (jetty-launcher-7-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@591dacdc{/solr,null,AVAILABLE}
[junit4] 2> 13092 INFO (jetty-launcher-7-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 13092 INFO (jetty-launcher-7-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 13093 INFO (jetty-launcher-7-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 13093 INFO (jetty-launcher-7-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@59d003e9{/solr,null,AVAILABLE}
[junit4] 2> 13094 INFO (jetty-launcher-7-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 13094 INFO (jetty-launcher-7-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 13094 INFO (jetty-launcher-7-thread-4) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 13094 INFO (jetty-launcher-7-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2882a621{/solr,null,AVAILABLE}
[junit4] 2> 13095 INFO (jetty-launcher-7-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 13095 INFO (jetty-launcher-7-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 13095 INFO (jetty-launcher-7-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 13096 INFO (jetty-launcher-7-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@447766b{/solr,null,AVAILABLE}
[junit4] 2> 13112 INFO (jetty-launcher-7-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@690ccd77{SSL,[ssl, http/1.1]}{127.0.0.1:43895}
[junit4] 2> 13113 INFO (jetty-launcher-7-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@55bb6c6b{SSL,[ssl, http/1.1]}{127.0.0.1:47372}
[junit4] 2> 13116 INFO (jetty-launcher-7-thread-3) [ ] o.e.j.s.Server Started @15405ms
[junit4] 2> 13116 INFO (jetty-launcher-7-thread-1) [ ] o.e.j.s.Server Started @15405ms
[junit4] 2> 13116 INFO (jetty-launcher-7-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43895}
[junit4] 2> 13116 INFO (jetty-launcher-7-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=47372}
[junit4] 2> 13116 ERROR (jetty-launcher-7-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 13116 ERROR (jetty-launcher-7-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 13116 INFO (jetty-launcher-7-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 13116 INFO (jetty-launcher-7-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 13116 INFO (jetty-launcher-7-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 13116 INFO (jetty-launcher-7-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 13116 INFO (jetty-launcher-7-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 13116 INFO (jetty-launcher-7-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 13117 INFO (jetty-launcher-7-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-23T00:24:42.852Z
[junit4] 2> 13117 INFO (jetty-launcher-7-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-23T00:24:42.853Z
[junit4] 2> 13117 INFO (jetty-launcher-7-thread-3) [ ] o.a.s.c.SolrResourceLoader solr home defaulted to 'solr/' (could not find system property or JNDI)
[junit4] 2> 13117 INFO (jetty-launcher-7-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@656e1dec{SSL,[ssl, http/1.1]}{127.0.0.1:46199}
[junit4] 2> 13117 INFO (jetty-launcher-7-thread-2) [ ] o.e.j.s.Server Started @15406ms
[junit4] 2> 13117 INFO (jetty-launcher-7-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46199}
[junit4] 2> 13118 ERROR (jetty-launcher-7-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 13118 INFO (jetty-launcher-7-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 13118 INFO (jetty-launcher-7-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 13118 INFO (jetty-launcher-7-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 13118 INFO (jetty-launcher-7-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-23T00:24:42.854Z
[junit4] 2> 13119 INFO (jetty-launcher-7-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1a8b69d{SSL,[ssl, http/1.1]}{127.0.0.1:40494}
[junit4] 2> 13119 INFO (jetty-launcher-7-thread-4) [ ] o.e.j.s.Server Started @15408ms
[junit4] 2> 13119 INFO (jetty-launcher-7-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40494}
[junit4] 2> 13119 ERROR (jetty-launcher-7-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 13120 INFO (jetty-launcher-7-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 13120 INFO (jetty-launcher-7-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 13120 INFO (jetty-launcher-7-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 13120 INFO (jetty-launcher-7-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-23T00:24:42.856Z
[junit4] 2> 13127 INFO (zkConnectionManagerCallback-12-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13136 INFO (zkConnectionManagerCallback-14-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13136 INFO (zkConnectionManagerCallback-16-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13136 INFO (jetty-launcher-7-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 13137 INFO (zkConnectionManagerCallback-18-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13140 INFO (jetty-launcher-7-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 13140 INFO (jetty-launcher-7-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 13141 INFO (jetty-launcher-7-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 13169 INFO (jetty-launcher-7-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58591/solr
[junit4] 2> 13174 INFO (jetty-launcher-7-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58591/solr
[junit4] 2> 13179 INFO (jetty-launcher-7-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58591/solr
[junit4] 2> 13184 INFO (jetty-launcher-7-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58591/solr
[junit4] 2> 13194 INFO (zkConnectionManagerCallback-34-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13199 INFO (zkConnectionManagerCallback-31-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13200 INFO (zkConnectionManagerCallback-28-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13200 INFO (zkConnectionManagerCallback-32-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13209 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1049018699f0005, likely client has closed socket
[junit4] 2> 13211 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1049018699f0008, likely client has closed socket
[junit4] 2> 13217 INFO (zkConnectionManagerCallback-37-thread-1-processing-n:127.0.0.1:47372_solr) [n:127.0.0.1:47372_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13224 INFO (zkConnectionManagerCallback-38-thread-1-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13229 INFO (zkConnectionManagerCallback-40-thread-1-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13244 INFO (zkConnectionManagerCallback-42-thread-1-processing-n:127.0.0.1:43895_solr) [n:127.0.0.1:43895_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 13326 WARN (jetty-launcher-7-thread-4) [n:127.0.0.1:40494_solr ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
[junit4] 2> 13326 WARN (jetty-launcher-7-thread-3) [n:127.0.0.1:43895_solr ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
[junit4] 2> 13327 WARN (jetty-launcher-7-thread-1) [n:127.0.0.1:47372_solr ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
[junit4] 2> 13329 WARN (jetty-launcher-7-thread-2) [n:127.0.0.1:46199_solr ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
[junit4] 2> 13357 INFO (jetty-launcher-7-thread-1) [n:127.0.0.1:47372_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 13357 INFO (jetty-launcher-7-thread-2) [n:127.0.0.1:46199_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 13358 INFO (jetty-launcher-7-thread-3) [n:127.0.0.1:43895_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 13359 INFO (jetty-launcher-7-thread-4) [n:127.0.0.1:40494_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 13360 INFO (jetty-launcher-7-thread-1) [n:127.0.0.1:47372_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:47372_solr
[junit4] 2> 13363 INFO (jetty-launcher-7-thread-3) [n:127.0.0.1:43895_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43895_solr
[junit4] 2> 13363 INFO (jetty-launcher-7-thread-4) [n:127.0.0.1:40494_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40494_solr
[junit4] 2> 13364 INFO (jetty-launcher-7-thread-2) [n:127.0.0.1:46199_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46199_solr
[junit4] 2> 13381 INFO (jetty-launcher-7-thread-1) [n:127.0.0.1:47372_solr ] o.a.s.c.Overseer Overseer (id=73341928470413321-127.0.0.1:47372_solr-n_0000000000) starting
[junit4] 2> 13421 INFO (zkCallback-41-thread-1-processing-n:127.0.0.1:43895_solr) [n:127.0.0.1:43895_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 13424 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 13455 INFO (zkCallback-39-thread-1-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 13456 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:47372_solr) [n:127.0.0.1:47372_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 13638 INFO (jetty-launcher-7-thread-1) [n:127.0.0.1:47372_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:47372_solr
[junit4] 2> 13640 INFO (zkCallback-39-thread-1-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 13640 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 13649 INFO (zkCallback-41-thread-1-processing-n:127.0.0.1:43895_solr) [n:127.0.0.1:43895_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 13649 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:47372_solr) [n:127.0.0.1:47372_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 13818 INFO (jetty-launcher-7-thread-3) [n:127.0.0.1:43895_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43895.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 13842 INFO (jetty-launcher-7-thread-2) [n:127.0.0.1:46199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46199.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 13900 INFO (jetty-launcher-7-thread-1) [n:127.0.0.1:47372_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47372.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 13952 INFO (jetty-launcher-7-thread-2) [n:127.0.0.1:46199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46199.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 13952 INFO (jetty-launcher-7-thread-2) [n:127.0.0.1:46199_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46199.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 13964 INFO (jetty-launcher-7-thread-3) [n:127.0.0.1:43895_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43895.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 13964 INFO (jetty-launcher-7-thread-3) [n:127.0.0.1:43895_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43895.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 13979 INFO (jetty-launcher-7-thread-4) [n:127.0.0.1:40494_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40494.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 13984 INFO (jetty-launcher-7-thread-1) [n:127.0.0.1:47372_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47372.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 13984 INFO (jetty-launcher-7-thread-1) [n:127.0.0.1:47372_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_47372.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 14002 INFO (jetty-launcher-7-thread-1) [n:127.0.0.1:47372_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.SelectWithEvaluatorsTest_32AE9B573185ED25-001/tempDir-001/node1/.
[junit4] 2> 14010 INFO (jetty-launcher-7-thread-4) [n:127.0.0.1:40494_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40494.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 14011 INFO (jetty-launcher-7-thread-4) [n:127.0.0.1:40494_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40494.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 14019 INFO (jetty-launcher-7-thread-3) [n:127.0.0.1:43895_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.SelectWithEvaluatorsTest_32AE9B573185ED25-001/tempDir-001/node3/.
[junit4] 2> 14042 INFO (jetty-launcher-7-thread-4) [n:127.0.0.1:40494_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.SelectWithEvaluatorsTest_32AE9B573185ED25-001/tempDir-001/node4/.
[junit4] 2> 14042 INFO (jetty-launcher-7-thread-2) [n:127.0.0.1:46199_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.SelectWithEvaluatorsTest_32AE9B573185ED25-001/tempDir-001/node2/.
[junit4] 2> 14437 INFO (zkConnectionManagerCallback-52-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 14452 INFO (zkConnectionManagerCallback-56-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 14454 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 14455 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58591/solr ready
[junit4] 2> 14723 INFO (qtp1305078931-140) [n:127.0.0.1:47372_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection1_collection&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 14772 INFO (OverseerThreadFactory-60-thread-1-processing-n:127.0.0.1:47372_solr) [n:127.0.0.1:47372_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1_collection
[junit4] 2> 14923 INFO (OverseerStateUpdate-73341928470413321-127.0.0.1:47372_solr-n_0000000000) [n:127.0.0.1:47372_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1_collection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_collection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:46199/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 14928 INFO (OverseerStateUpdate-73341928470413321-127.0.0.1:47372_solr-n_0000000000) [n:127.0.0.1:47372_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1_collection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1_collection_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40494/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 15258 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_collection_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 15259 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 15263 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_collection_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 15263 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 15400 INFO (zkCallback-39-thread-1-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 15400 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 15415 INFO (zkCallback-39-thread-2-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 15415 INFO (zkCallback-36-thread-2-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 16313 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 16318 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 16334 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.s.IndexSchema [collection1_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 16357 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.s.IndexSchema [collection1_collection_shard2_replica_n2] Schema name=test
[junit4] 2> 16485 WARN (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.GeoHashField]. Please consult documentation how to replace it accordingly.
[junit4] 2> 16489 WARN (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.SolrResourceLoader Solr loaded a deprecated plugin/analysis class [solr.LatLonType]. Please consult documentation how to replace it accordingly.
[junit4] 2> 16502 WARN (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 16502 WARN (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 16510 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 16513 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_collection_shard1_replica_n1' using configuration from collection collection1_collection, trusted=true
[junit4] 2> 16515 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46199.solr.core.collection1_collection.shard1.replica_n1' (registry 'solr.core.collection1_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 16515 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 16515 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.SolrCore [[collection1_collection_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.SelectWithEvaluatorsTest_32AE9B573185ED25-001/tempDir-001/node2/collection1_collection_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.SelectWithEvaluatorsTest_32AE9B573185ED25-001/tempDir-001/node2/./collection1_collection_shard1_replica_n1/data/]
[junit4] 2> 16522 WARN (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.s.IndexSchema Field lowerfilt1and2 is not multivalued and destination for multiple copyFields (2)
[junit4] 2> 16522 WARN (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.s.IndexSchema Field text is not multivalued and destination for multiple copyFields (3)
[junit4] 2> 16523 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 16523 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_collection_shard2_replica_n2' using configuration from collection collection1_collection, trusted=true
[junit4] 2> 16523 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40494.solr.core.collection1_collection.shard2.replica_n2' (registry 'solr.core.collection1_collection.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@75433683
[junit4] 2> 16523 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 16523 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.SolrCore [[collection1_collection_shard2_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.SelectWithEvaluatorsTest_32AE9B573185ED25-001/tempDir-001/node4/collection1_collection_shard2_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.io.stream.SelectWithEvaluatorsTest_32AE9B573185ED25-001/tempDir-001/node4/./collection1_collection_shard2_replica_n2/data/]
[junit4] 2> 16592 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 16592 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 16593 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 16593 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 16597 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@5a48b104[collection1_collection_shard1_replica_n1] main]
[junit4] 2> 16600 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 16600 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 16601 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 16601 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 16602 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 16605 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 16605 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 16605 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593149212685500416
[junit4] 2> 16607 INFO (searcherExecutor-68-thread-1-processing-n:127.0.0.1:46199_solr x:collection1_collection_shard1_replica_n1 s:shard1 c:collection1_collection r:core_node3) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.SolrCore [collection1_collection_shard1_replica_n1] Registered new searcher Searcher@5a48b104[collection1_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 16608 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@3c6dec8c[collection1_collection_shard2_replica_n2] main]
[junit4] 2> 16610 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 16615 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 16615 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 16623 INFO (searcherExecutor-69-thread-1-processing-n:127.0.0.1:40494_solr x:collection1_collection_shard2_replica_n2 s:shard2 c:collection1_collection r:core_node4) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.SolrCore [collection1_collection_shard2_replica_n2] Registered new searcher Searcher@3c6dec8c[collection1_collection_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 16624 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593149212705423360
[junit4] 2> 16648 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 16649 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 16650 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:40494/solr/collection1_collection_shard2_replica_n2/
[junit4] 2> 16651 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 16651 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:40494/solr/collection1_collection_shard2_replica_n2/ has no replicas
[junit4] 2> 16651 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 16648 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 16656 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 16656 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:46199/solr/collection1_collection_shard1_replica_n1/
[junit4] 2> 16656 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 16656 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:46199/solr/collection1_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 16656 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 16665 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:46199/solr/collection1_collection_shard1_replica_n1/ shard1
[junit4] 2> 16665 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:40494/solr/collection1_collection_shard2_replica_n2/ shard2
[junit4] 2> 16768 INFO (zkCallback-39-thread-1-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 16768 INFO (zkCallback-39-thread-2-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 16768 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 16768 INFO (zkCallback-36-thread-2-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 16817 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 16817 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 16823 INFO (qtp1320716203-134) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_collection_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1_collection&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1566
[junit4] 2> 16824 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_collection_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1561
[junit4] 2> 16842 INFO (qtp1305078931-140) [n:127.0.0.1:47372_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 16923 INFO (zkCallback-39-thread-1-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 16923 INFO (zkCallback-39-thread-2-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 16923 INFO (zkCallback-36-thread-1-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 16923 INFO (zkCallback-36-thread-2-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1_collection/state.json] for collection [collection1_collection] has occurred - updating... (live nodes size: [4])
[junit4] 2> 17844 INFO (qtp1305078931-140) [n:127.0.0.1:47372_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection1_collection&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=3120
[junit4] 2> 17851 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1_collection failOnTimeout:true timeout (sec):90
[junit4] 2> 17852 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1_collection
[junit4] 2> 17859 INFO (qtp1305078931-142) [n:127.0.0.1:47372_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :createalias with params collections=collection1_collection&name=collection1&action=CREATEALIAS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 17876 INFO (OverseerCollectionConfigSetProcessor-73341928470413321-127.0.0.1:47372_solr-n_0000000000) [n:127.0.0.1:47372_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 18003 INFO (qtp1305078931-142) [n:127.0.0.1:47372_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={collections=collection1_collection&name=collection1&action=CREATEALIAS&wt=javabin&version=2} status=0 QTime=144
[junit4] 2> 18052 INFO (TEST-SelectWithEvaluatorsTest.testSelectWithEvaluatorsStream-seed#[32AE9B573185ED25]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSelectWithEvaluatorsStream
[junit4] 2> 18197 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_collection_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=https://127.0.0.1:40494/solr/collection1_collection_shard2_replica_n2/&wt=javabin&version=2}{deleteByQuery=*:* (-1593149214353784832)} 0 1
[junit4] 2> 18237 INFO (qtp833128252-128) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1593149214396776448,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 18237 INFO (qtp833128252-128) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 18238 INFO (qtp833128252-128) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 18248 INFO (qtp833128252-128) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_collection_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:40494/solr/collection1_collection_shard2_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 11
[junit4] 2> 18267 INFO (qtp1320716203-135) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1593149214428233728,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 18267 INFO (qtp1320716203-135) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 18272 INFO (qtp1320716203-135) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 18272 INFO (qtp1320716203-135) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_collection_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:40494/solr/collection1_collection_shard2_replica_n2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 5
[junit4] 2> 18273 INFO (qtp1320716203-137) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_collection_shard2_replica_n2] webapp=/solr path=/update params={_stateVer_=collection1_collection:4&commit=true&wt=javabin&version=2}{deleteByQuery=*:* (-1593149214261510144),commit=} 0 182
[junit4] 2> 18366 INFO (qtp833128252-127) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_collection_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=collection1_collection:4&wt=javabin&version=2}{add=[1 (1593149214481711104)]} 0 49
[junit4] 2> 18408 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1593149214576082944,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 18412 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@4ac011f1 commitCommandVersion:1593149214576082944
[junit4] 2> 18421 INFO (qtp1320716203-137) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1593149214589714432,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 18425 INFO (qtp1320716203-137) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 18425 INFO (qtp1320716203-137) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 18426 INFO (qtp1320716203-137) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_collection_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:46199/solr/collection1_collection_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 5
[junit4] 2> 18459 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@29b0233e[collection1_collection_shard1_replica_n1] main]
[junit4] 2> 18460 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 18462 INFO (searcherExecutor-68-thread-1-processing-n:127.0.0.1:46199_solr x:collection1_collection_shard1_replica_n1 s:shard1 c:collection1_collection r:core_node3) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.SolrCore [collection1_collection_shard1_replica_n1] Registered new searcher Searcher@29b0233e[collection1_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C1)))}
[junit4] 2> 18462 INFO (qtp833128252-125) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_collection_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:46199/solr/collection1_collection_shard1_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 54
[junit4] 2> 18469 INFO (qtp833128252-127) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_collection_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=collection1_collection:4&commit=true&wt=javabin&version=2}{commit=} 0 98
[junit4] 2> 18495 INFO (zkConnectionManagerCallback-60-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 18506 INFO (TEST-SelectWithEvaluatorsTest.testSelectWithEvaluatorsStream-seed#[32AE9B573185ED25]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 18507 INFO (TEST-SelectWithEvaluatorsTest.testSelectWithEvaluatorsStream-seed#[32AE9B573185ED25]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58591/solr ready
[junit4] 2> 18584 INFO (qtp1320716203-138) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.S.Request [collection1_collection_shard2_replica_n2] webapp=/solr path=/select params={q=*:*&distrib=false&fl=id,a_s,b_i,c_d,d_b&sort=id+asc&wt=json&version=2.2} hits=0 status=0 QTime=12
[junit4] 2> 18608 INFO (qtp833128252-128) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.S.Request [collection1_collection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&distrib=false&fl=id,a_s,b_i,c_d,d_b&sort=id+asc&wt=json&version=2.2} hits=1 status=0 QTime=36
[junit4] 2> 18618 INFO (TEST-SelectWithEvaluatorsTest.testSelectWithEvaluatorsStream-seed#[32AE9B573185ED25]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSelectWithEvaluatorsStream
[junit4] 2> 18704 INFO (jetty-closer-8-thread-4) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@656e1dec{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 18704 INFO (jetty-closer-8-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@55bb6c6b{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 18708 INFO (jetty-closer-8-thread-4) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=373308328
[junit4] 2> 18709 INFO (jetty-closer-8-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=594140042
[junit4] 2> 18719 INFO (jetty-closer-8-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@690ccd77{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 18719 INFO (jetty-closer-8-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 18736 INFO (jetty-closer-8-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 18736 INFO (jetty-closer-8-thread-3) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1a8b69d{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 18755 INFO (jetty-closer-8-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=703365673
[junit4] 2> 18755 INFO (jetty-closer-8-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@188327ad: rootName = solr_47372, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@12e76644
[junit4] 2> 18755 INFO (jetty-closer-8-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 18756 INFO (jetty-closer-8-thread-4) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@50f06e67: rootName = solr_46199, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@befb435
[junit4] 2> 18762 INFO (jetty-closer-8-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@222fb2f9: rootName = solr_43895, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5160a373
[junit4] 2> 18762 INFO (jetty-closer-8-thread-3) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=759804559
[junit4] 2> 18763 INFO (jetty-closer-8-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 18786 INFO (jetty-closer-8-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 18786 INFO (jetty-closer-8-thread-4) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2d744eb4: rootName = solr_46199, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5285b38
[junit4] 2> 18793 INFO (jetty-closer-8-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 18793 INFO (jetty-closer-8-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2b5a0697: rootName = solr_47372, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5285b38
[junit4] 2> 18798 INFO (jetty-closer-8-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 18798 INFO (jetty-closer-8-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1f88cc01: rootName = solr_43895, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5285b38
[junit4] 2> 18820 INFO (jetty-closer-8-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 18820 INFO (jetty-closer-8-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1a657322: rootName = solr_43895, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@5fa7472f
[junit4] 2> 18822 INFO (jetty-closer-8-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 18822 INFO (jetty-closer-8-thread-4) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@24808fac: rootName = solr_46199, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@5fa7472f
[junit4] 2> 18824 INFO (jetty-closer-8-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 18824 INFO (jetty-closer-8-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@71507c02: rootName = solr_47372, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@5fa7472f
[junit4] 2> 18824 INFO (jetty-closer-8-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5fb0a51: rootName = solr_40494, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@79904280
[junit4] 2> 18826 INFO (jetty-closer-8-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:43895_solr
[junit4] 2> 18828 INFO (zkCallback-39-thread-2-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 18828 INFO (zkCallback-35-thread-1-processing-n:127.0.0.1:47372_solr) [n:127.0.0.1:47372_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 18828 INFO (jetty-closer-8-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 18828 INFO (jetty-closer-8-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 18830 INFO (zkCallback-41-thread-1-processing-n:127.0.0.1:43895_solr) [n:127.0.0.1:43895_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 18831 INFO (jetty-closer-8-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 18831 INFO (jetty-closer-8-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6dcce2ef: rootName = solr_40494, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5285b38
[junit4] 2> 18835 INFO (jetty-closer-8-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 18835 INFO (jetty-closer-8-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5741cae7: rootName = solr_40494, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@5fa7472f
[junit4] 2> 18842 INFO (zkCallback-36-thread-2-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 18847 INFO (jetty-closer-8-thread-4) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:46199_solr
[junit4] 2> 18848 INFO (jetty-closer-8-thread-4) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 18848 INFO (zkCallback-36-thread-2-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 18855 INFO (jetty-closer-8-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:47372_solr
[junit4] 2> 18855 INFO (zkCallback-39-thread-1-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 18856 INFO (jetty-closer-8-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 18856 INFO (jetty-closer-8-thread-2) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 18857 INFO (zkCallback-39-thread-2-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 18857 INFO (zkCallback-36-thread-2-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 18860 INFO (jetty-closer-8-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@591dacdc{/solr,null,UNAVAILABLE}
[junit4] 2> 18867 INFO (jetty-closer-8-thread-3) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40494_solr
[junit4] 2> 18868 INFO (zkCallback-36-thread-2-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 18869 INFO (zkCallback-39-thread-2-processing-n:127.0.0.1:40494_solr) [n:127.0.0.1:40494_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 18869 INFO (jetty-closer-8-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 18904 INFO (jetty-closer-8-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 18912 INFO (coreCloseExecutor-80-thread-1) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.c.SolrCore [collection1_collection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@12b22195
[junit4] 2> 18912 INFO (zkCallback-35-thread-2-processing-n:127.0.0.1:47372_solr) [n:127.0.0.1:47372_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (0)
[junit4] 2> 18913 INFO (coreCloseExecutor-80-thread-1) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1_collection.shard1.replica_n1, tag=313663893
[junit4] 2> 18916 INFO (coreCloseExecutor-80-thread-1) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@d2c0343: rootName = solr_46199, domain = solr.core.collection1_collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection1_collection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@29d43924
[junit4] 2> 18923 INFO (coreCloseExecutor-81-thread-1) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.c.SolrCore [collection1_collection_shard2_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@35babd4c
[junit4] 2> 18923 INFO (coreCloseExecutor-81-thread-1) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1_collection.shard2.replica_n2, tag=901430604
[junit4] 2> 18923 INFO (coreCloseExecutor-81-thread-1) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@74e25f4f: rootName = solr_40494, domain = solr.core.collection1_collection.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.collection1_collection.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@15b888c7
[junit4] 2> 18936 INFO (jetty-closer-8-thread-2) [ ] o.a.s.c.Overseer Overseer (id=73341928470413321-127.0.0.1:47372_solr-n_0000000000) closing
[junit4] 2> 18937 INFO (OverseerStateUpdate-73341928470413321-127.0.0.1:47372_solr-n_0000000000) [n:127.0.0.1:47372_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:47372_solr
[junit4] 2> 18941 WARN (OverseerAutoScalingTriggerThread-73341928470413321-127.0.0.1:47372_solr-n_0000000000) [n:127.0.0.1:47372_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 18957 INFO (jetty-closer-8-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@447766b{/solr,null,UNAVAILABLE}
[junit4] 2> 18954 INFO (zkCallback-36-thread-2-processing-n:127.0.0.1:46199_solr) [n:127.0.0.1:46199_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:46199_solr
[junit4] 2> 18959 INFO (jetty-closer-8-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 18987 INFO (coreCloseExecutor-81-thread-1) [n:127.0.0.1:40494_solr c:collection1_collection s:shard2 r:core_node4 x:collection1_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1_collection.shard2.leader, tag=901430604
[junit4] 2> 18990 INFO (coreCloseExecutor-80-thread-1) [n:127.0.0.1:46199_solr c:collection1_collection s:shard1 r:core_node3 x:collection1_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1_collection.shard1.leader, tag=313663893
[junit4] 2> 19023 INFO (jetty-closer-8-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2882a621{/solr,null,UNAVAILABLE}
[junit4] 2> 19026 INFO (jetty-closer-8-thread-3) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 19032 INFO (jetty-closer-8-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@59d003e9{/solr,null,UNAVAILABLE}
[junit4] 2> 19032 INFO (jetty-closer-8-thread-4) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 49026 WARN (jetty-closer-8-thread-3) [ ] o.e.j.u.t.QueuedThreadPool QueuedThreadPool@qtp1320716203{STOPPING,8<=8<=10000,i=0,q=1} Couldn't stop Thread[qtp1320716203-134,5,TGRP-SelectWithEvaluatorsTest]
[junit4] 2> 49027 ERROR (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 49028 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58591 58591
[junit4] 2> 49229 INFO (Thread-31) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58591 58591
[junit4] 2> 49230 WARN (Thread-31) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/aliases.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 2 /solr/configs/conf
[junit4] 2> 2 /solr/collections/collection1_collection/terms/shard1
[junit4] 2> 2 /solr/collections/collection1_collection/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/collections/collection1_collection/state.json
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2> 2 /solr/overseer_elect/election/73341928470413322-127.0.0.1:46199_solr-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/live_nodes
[junit4] 2> 6 /solr/collections
[junit4] 2>
[junit4] 2> 49240 INFO (SUITE-SelectWithEvaluatorsTest-seed#[32AE9B573185ED25]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 23, 2018 12:25:18 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 23, 2018 12:25:39 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.io.stream.SelectWithEvaluatorsTest:
[junit4] 2> 1) Thread[id=134, name=qtp1320716203-134, state=TIMED_WAITING, group=TGRP-SelectWithEvaluatorsTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 23, 2018 12:25:39 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=134, name=qtp1320716203-134, state=TIMED_WAITING, group=TGRP-SelectWithEvaluatorsTest]
[junit4] 2> Feb 23, 2018 12:25:42 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=134, name=qtp1320716203-134, state=TIMED_WAITING, group=TGRP-SelectWithEvaluatorsTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2df58717), locale=mk, timezone=Asia/Kuching
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=2,free=204712360,total=307232768
[junit4] 2> NOTE: All tests run in this JVM: [LargeVolumeJettyTest, NormalizeEvaluatorTest, AddEvaluatorTest, ExclusiveOrEvaluatorTest, GetByIdTest, TestBatchUpdate, MultiplyEvaluatorTest, TestValidatingJsonMap, SelectWithEvaluatorsTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=SelectWithEvaluatorsTest -Dtests.seed=32AE9B573185ED25 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=mk -Dtests.timezone=Asia/Kuching -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 0.00s J2 | SelectWithEvaluatorsTest (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.io.stream.SelectWithEvaluatorsTest:
[junit4] > 1) Thread[id=134, name=qtp1320716203-134, state=TIMED_WAITING, group=TGRP-SelectWithEvaluatorsTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([32AE9B573185ED25]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=134, name=qtp1320716203-134, state=TIMED_WAITING, group=TGRP-SelectWithEvaluatorsTest]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([32AE9B573185ED25]:0)
[junit4] Completed [79/146 (1!)] on J2 in 61.25s, 1 test, 2 errors <<< FAILURES!
[...truncated 49328 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2360 - Still Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2360/
5 tests failed.
FAILED: org.apache.solr.cloud.ReplaceNodeNoTargetTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([95AAE7548AC99BF:810E91AFE650F447]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue
Error Message:
action wasn't interrupted
Stack Trace:
java.lang.AssertionError: action wasn't interrupted
at __randomizedtesting.SeedInfo.seed([95AAE7548AC99BF:C0EFECDB41CB5F4A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testEventQueue(TriggerIntegrationTest.java:726)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory
Error Message:
expected:<5> but was:<0>
Stack Trace:
java.lang.AssertionError: expected:<5> but was:<0>
at __randomizedtesting.SeedInfo.seed([95AAE7548AC99BF:64A60A88F2E466B8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.admin.AutoscalingHistoryHandlerTest.testHistory(AutoscalingHistoryHandlerTest.java:313)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.request.TestV2Request
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.client.solrj.request.TestV2Request: 1) Thread[id=93, name=qtp1429719128-93, state=TIMED_WAITING, group=TGRP-TestV2Request] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.request.TestV2Request:
1) Thread[id=93, name=qtp1429719128-93, state=TIMED_WAITING, group=TGRP-TestV2Request]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([1A3FDF5A1DEC7E28]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.request.TestV2Request
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=93, name=qtp1429719128-93, state=TIMED_WAITING, group=TGRP-TestV2Request] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=93, name=qtp1429719128-93, state=TIMED_WAITING, group=TGRP-TestV2Request]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([1A3FDF5A1DEC7E28]:0)
Build Log:
[...truncated 12192 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> 108282 INFO (SUITE-TriggerIntegrationTest-seed#[95AAE7548AC99BF]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/init-core-data-001
[junit4] 2> 108354 WARN (SUITE-TriggerIntegrationTest-seed#[95AAE7548AC99BF]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 108355 INFO (SUITE-TriggerIntegrationTest-seed#[95AAE7548AC99BF]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 108356 INFO (SUITE-TriggerIntegrationTest-seed#[95AAE7548AC99BF]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 108410 INFO (SUITE-TriggerIntegrationTest-seed#[95AAE7548AC99BF]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/tempDir-001
[junit4] 2> 108410 INFO (SUITE-TriggerIntegrationTest-seed#[95AAE7548AC99BF]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 108414 INFO (Thread-41) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 108414 INFO (Thread-41) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 108435 ERROR (Thread-41) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 108523 INFO (SUITE-TriggerIntegrationTest-seed#[95AAE7548AC99BF]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:39596
[junit4] 2> 108682 INFO (zkConnectionManagerCallback-64-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 108724 INFO (jetty-launcher-61-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 108781 INFO (jetty-launcher-61-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 108849 INFO (jetty-launcher-61-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 108849 INFO (jetty-launcher-61-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 108849 INFO (jetty-launcher-61-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 108850 INFO (jetty-launcher-61-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@12fb50cb{/solr,null,AVAILABLE}
[junit4] 2> 108942 INFO (jetty-launcher-61-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 108942 INFO (jetty-launcher-61-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 108942 INFO (jetty-launcher-61-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 108978 INFO (jetty-launcher-61-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@d4b1038{/solr,null,AVAILABLE}
[junit4] 2> 108995 INFO (jetty-launcher-61-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@765b4865{HTTP/1.1,[http/1.1]}{127.0.0.1:33129}
[junit4] 2> 108995 INFO (jetty-launcher-61-thread-1) [ ] o.e.j.s.Server Started @119728ms
[junit4] 2> 108995 INFO (jetty-launcher-61-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33129}
[junit4] 2> 108995 ERROR (jetty-launcher-61-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 108995 INFO (jetty-launcher-61-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 108995 INFO (jetty-launcher-61-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 108995 INFO (jetty-launcher-61-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 108995 INFO (jetty-launcher-61-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T20:22:29.426Z
[junit4] 2> 108996 INFO (jetty-launcher-61-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@74a8be89{HTTP/1.1,[http/1.1]}{127.0.0.1:45394}
[junit4] 2> 108996 INFO (jetty-launcher-61-thread-2) [ ] o.e.j.s.Server Started @119729ms
[junit4] 2> 108996 INFO (jetty-launcher-61-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45394}
[junit4] 2> 108996 ERROR (jetty-launcher-61-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 108996 INFO (jetty-launcher-61-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 108996 INFO (jetty-launcher-61-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 108996 INFO (jetty-launcher-61-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 108996 INFO (jetty-launcher-61-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T20:22:29.427Z
[junit4] 2> 109185 INFO (zkConnectionManagerCallback-66-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 109209 INFO (zkConnectionManagerCallback-68-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 109259 INFO (jetty-launcher-61-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 109259 INFO (jetty-launcher-61-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 109545 INFO (jetty-launcher-61-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39596/solr
[junit4] 2> 109657 INFO (jetty-launcher-61-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39596/solr
[junit4] 2> 109658 INFO (zkConnectionManagerCallback-72-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 109962 INFO (zkConnectionManagerCallback-76-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 110010 INFO (zkConnectionManagerCallback-78-thread-1-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 110271 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1006370ff6a0004, likely client has closed socket
[junit4] 2> 110394 INFO (zkConnectionManagerCallback-80-thread-1-processing-n:127.0.0.1:33129_solr) [n:127.0.0.1:33129_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 110948 INFO (jetty-launcher-61-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 110984 INFO (jetty-launcher-61-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33129_solr
[junit4] 2> 111207 INFO (jetty-launcher-61-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.c.Overseer Overseer (id=72166931010551814-127.0.0.1:33129_solr-n_0000000000) starting
[junit4] 2> 111810 INFO (jetty-launcher-61-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33129_solr
[junit4] 2> 111810 DEBUG (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 111846 DEBUG (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 111867 DEBUG (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: []
[junit4] 2> 111867 DEBUG (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 111867 DEBUG (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 111997 INFO (OverseerStateUpdate-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 112035 DEBUG (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 112065 DEBUG (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 112114 DEBUG (ScheduledTrigger-128-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 112505 INFO (jetty-launcher-61-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33129.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 112581 INFO (jetty-launcher-61-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33129.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 112598 INFO (jetty-launcher-61-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33129.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 112615 INFO (jetty-launcher-61-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/tempDir-001/node1/.
[junit4] 2> 113216 DEBUG (ScheduledTrigger-128-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 113531 INFO (jetty-launcher-61-thread-2) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 113551 INFO (jetty-launcher-61-thread-2) [n:127.0.0.1:45394_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 113603 INFO (jetty-launcher-61-thread-2) [n:127.0.0.1:45394_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45394_solr
[junit4] 2> 113676 INFO (zkCallback-79-thread-2-processing-n:127.0.0.1:33129_solr) [n:127.0.0.1:33129_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 113796 INFO (zkCallback-77-thread-1-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 114455 DEBUG (ScheduledTrigger-128-thread-2) [n:127.0.0.1:33129_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 114672 INFO (jetty-launcher-61-thread-2) [n:127.0.0.1:45394_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45394.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 114818 INFO (jetty-launcher-61-thread-2) [n:127.0.0.1:45394_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45394.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 114818 INFO (jetty-launcher-61-thread-2) [n:127.0.0.1:45394_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45394.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 114819 INFO (jetty-launcher-61-thread-2) [n:127.0.0.1:45394_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/tempDir-001/node2/.
[junit4] 2> 115377 INFO (zkConnectionManagerCallback-86-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 115597 DEBUG (ScheduledTrigger-128-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 115604 INFO (zkConnectionManagerCallback-90-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 115615 INFO (SUITE-TriggerIntegrationTest-seed#[95AAE7548AC99BF]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 115616 INFO (SUITE-TriggerIntegrationTest-seed#[95AAE7548AC99BF]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39596/solr ready
[junit4] 2> 116453 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
[junit4] 2> 116510 INFO (qtp131808664-261) [n:127.0.0.1:45394_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 116727 DEBUG (ScheduledTrigger-128-thread-1) [n:127.0.0.1:33129_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 117090 INFO (qtp131808664-261) [n:127.0.0.1:45394_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=579
[junit4] 2> 117234 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: .auto_add_replicas
[junit4] 2> 117254 DEBUG (zkCallback-79-thread-2-processing-n:127.0.0.1:33129_solr) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 117254 DEBUG (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 117254 DEBUG (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 117255 DEBUG (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 117255 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 117256 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 117257 DEBUG (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 117327 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@765b4865{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 117396 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1902405966
[junit4] 2> 117424 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 117432 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7b6e8c24: rootName = solr_33129, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@389e0560
[junit4] 2> 117485 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 117485 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@369f1385: rootName = solr_33129, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@61c05b9e
[junit4] 2> 117508 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 117509 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3a509c6a: rootName = solr_33129, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@35dd9c92
[junit4] 2> 117522 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33129_solr
[junit4] 2> 117523 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 117523 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 117524 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.Overseer Overseer (id=72166931010551814-127.0.0.1:33129_solr-n_0000000000) closing
[junit4] 2> 117526 INFO (OverseerStateUpdate-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:33129_solr
[junit4] 2> 117558 INFO (zkCallback-79-thread-1-processing-n:127.0.0.1:33129_solr) [n:127.0.0.1:33129_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 117565 WARN (OverseerAutoScalingTriggerThread-72166931010551814-127.0.0.1:33129_solr-n_0000000000) [n:127.0.0.1:33129_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 117708 DEBUG (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 117745 INFO (zkCallback-77-thread-1-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 117870 INFO (zkCallback-89-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 117871 INFO (zkCallback-89-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 117891 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1006370ff6a0006, likely client has closed socket
[junit4] 2> 117915 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@12fb50cb{/solr,null,UNAVAILABLE}
[junit4] 2> 117917 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 117931 INFO (zkCallback-77-thread-1-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45394_solr
[junit4] 2> 117949 INFO (zkCallback-77-thread-1-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.Overseer Overseer (id=72166931010551813-127.0.0.1:45394_solr-n_0000000001) starting
[junit4] 2> 117981 DEBUG (OverseerAutoScalingTriggerThread-72166931010551813-127.0.0.1:45394_solr-n_0000000001) [n:127.0.0.1:45394_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 117982 DEBUG (OverseerAutoScalingTriggerThread-72166931010551813-127.0.0.1:45394_solr-n_0000000001) [n:127.0.0.1:45394_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 117983 DEBUG (OverseerAutoScalingTriggerThread-72166931010551813-127.0.0.1:45394_solr-n_0000000001) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:45394_solr]
[junit4] 2> 117983 DEBUG (OverseerAutoScalingTriggerThread-72166931010551813-127.0.0.1:45394_solr-n_0000000001) [n:127.0.0.1:45394_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion -1
[junit4] 2> 117983 DEBUG (OverseerAutoScalingTriggerThread-72166931010551813-127.0.0.1:45394_solr-n_0000000001) [n:127.0.0.1:45394_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 118006 DEBUG (OverseerAutoScalingTriggerThread-72166931010551813-127.0.0.1:45394_solr-n_0000000001) [n:127.0.0.1:45394_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 118006 DEBUG (OverseerAutoScalingTriggerThread-72166931010551813-127.0.0.1:45394_solr-n_0000000001) [n:127.0.0.1:45394_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 118007 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 119016 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 120024 DEBUG (ScheduledTrigger-137-thread-2) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 121035 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 122039 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 122937 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 122938 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 122938 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 122938 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 122966 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7f4798c8{/solr,null,AVAILABLE}
[junit4] 2> 122967 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6fd42303{HTTP/1.1,[http/1.1]}{127.0.0.1:36772}
[junit4] 2> 122967 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.Server Started @133701ms
[junit4] 2> 122967 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=36772}
[junit4] 2> 122967 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 122968 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 122968 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 122968 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 122969 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T20:22:43.400Z
[junit4] 2> 123032 INFO (zkConnectionManagerCallback-93-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 123033 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 123051 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 123187 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39596/solr
[junit4] 2> 123246 INFO (zkConnectionManagerCallback-97-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 123384 INFO (zkConnectionManagerCallback-99-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 123429 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 123431 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:36772_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 123434 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:36772_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36772_solr
[junit4] 2> 123435 INFO (zkCallback-89-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 123435 INFO (zkCallback-89-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 123490 INFO (zkCallback-77-thread-2-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 123519 INFO (zkCallback-98-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 123930 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:36772_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36772.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 124057 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:36772_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36772.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 124058 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:36772_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36772.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 124067 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 124113 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:36772_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/tempDir-001/node3/.
[junit4] 2> 124448 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 124449 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 124469 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 124469 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 124494 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7ad630d5{/solr,null,AVAILABLE}
[junit4] 2> 124495 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6e8d7408{HTTP/1.1,[http/1.1]}{127.0.0.1:40499}
[junit4] 2> 124495 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.Server Started @135228ms
[junit4] 2> 124495 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40499}
[junit4] 2> 124495 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 124495 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 124495 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 124495 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 124495 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T20:22:44.926Z
[junit4] 2> 124517 INFO (zkConnectionManagerCallback-103-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 124530 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 124594 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39596/solr
[junit4] 2> 124639 INFO (zkConnectionManagerCallback-107-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 124657 INFO (zkConnectionManagerCallback-109-thread-1-processing-n:127.0.0.1:40499_solr) [n:127.0.0.1:40499_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 124774 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:40499_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 124777 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:40499_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 124881 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:40499_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40499_solr
[junit4] 2> 124883 INFO (zkCallback-77-thread-2-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 124883 INFO (zkCallback-89-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 124883 INFO (zkCallback-89-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 124883 INFO (zkCallback-98-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 124968 INFO (zkCallback-108-thread-1-processing-n:127.0.0.1:40499_solr) [n:127.0.0.1:40499_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 125084 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 125452 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:40499_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40499.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 125705 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:40499_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40499.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 125721 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:40499_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40499.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 125724 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:40499_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/tempDir-001/node4/.
[junit4] 2> 126052 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 126064 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 126064 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 126064 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 126065 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2a663d30{/solr,null,AVAILABLE}
[junit4] 2> 126065 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5bb71271{HTTP/1.1,[http/1.1]}{127.0.0.1:44881}
[junit4] 2> 126065 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.Server Started @136799ms
[junit4] 2> 126065 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44881}
[junit4] 2> 126078 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 126078 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 126078 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 126078 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 126078 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T20:22:46.509Z
[junit4] 2> 126112 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 126162 INFO (zkConnectionManagerCallback-113-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 126163 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 126213 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39596/solr
[junit4] 2> 126302 INFO (zkConnectionManagerCallback-117-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 126331 INFO (zkConnectionManagerCallback-119-thread-1-processing-n:127.0.0.1:44881_solr) [n:127.0.0.1:44881_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 126391 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:44881_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 126394 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:44881_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 126396 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:44881_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44881_solr
[junit4] 2> 126419 INFO (zkCallback-98-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 126427 INFO (zkCallback-77-thread-1-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 126431 INFO (zkCallback-89-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 126432 INFO (zkCallback-89-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 126432 INFO (zkCallback-108-thread-1-processing-n:127.0.0.1:40499_solr) [n:127.0.0.1:40499_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 126496 INFO (zkCallback-118-thread-1-processing-n:127.0.0.1:44881_solr) [n:127.0.0.1:44881_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 126959 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:44881_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44881.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 127104 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:44881_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44881.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 127104 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:44881_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44881.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 127120 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 127159 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:44881_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/tempDir-001/node5/.
[junit4] 2> 127498 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 127528 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 127577 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 127578 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 127586 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@23754aad{/solr,null,AVAILABLE}
[junit4] 2> 127586 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@777893fc{HTTP/1.1,[http/1.1]}{127.0.0.1:38196}
[junit4] 2> 127586 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.e.j.s.Server Started @138320ms
[junit4] 2> 127587 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38196}
[junit4] 2> 127587 ERROR (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 127587 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 127587 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 127587 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 127587 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T20:22:48.018Z
[junit4] 2> 127638 INFO (zkConnectionManagerCallback-123-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 127643 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 127713 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39596/solr
[junit4] 2> 127768 INFO (zkConnectionManagerCallback-127-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 127858 INFO (zkConnectionManagerCallback-129-thread-1-processing-n:127.0.0.1:38196_solr) [n:127.0.0.1:38196_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 127914 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:38196_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 127928 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:38196_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 127930 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:38196_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38196_solr
[junit4] 2> 127984 INFO (zkCallback-98-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 127984 INFO (zkCallback-89-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 127984 INFO (zkCallback-89-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 128073 INFO (zkCallback-128-thread-1-processing-n:127.0.0.1:38196_solr) [n:127.0.0.1:38196_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 128116 INFO (zkCallback-77-thread-1-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 128120 INFO (zkCallback-108-thread-1-processing-n:127.0.0.1:40499_solr) [n:127.0.0.1:40499_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 128154 INFO (zkCallback-118-thread-1-processing-n:127.0.0.1:44881_solr) [n:127.0.0.1:44881_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 128166 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 128330 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:38196_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38196.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 128369 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:38196_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38196.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 128386 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:38196_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38196.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 128387 INFO (TEST-TriggerIntegrationTest.testSearchRate-seed#[95AAE7548AC99BF]) [n:127.0.0.1:38196_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/tempDir-001/node6/.
[junit4] 2> 128752 INFO (qtp1365257422-337) [n:127.0.0.1:36772_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&name=collection1&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 128918 INFO (OverseerThreadFactory-139-thread-1-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 129216 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 129398 INFO (OverseerStateUpdate-72166931010551813-127.0.0.1:45394_solr-n_0000000001) [n:127.0.0.1:45394_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:45394/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 129399 INFO (OverseerStateUpdate-72166931010551813-127.0.0.1:45394_solr-n_0000000001) [n:127.0.0.1:45394_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:36772/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 130040 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 130041 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 130058 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n2&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 130058 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 130185 INFO (zkCallback-98-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 130186 INFO (zkCallback-98-thread-2-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 130222 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 130284 INFO (zkCallback-77-thread-1-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 130295 INFO (zkCallback-98-thread-2-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 130295 INFO (zkCallback-98-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 131072 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 131118 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema [collection1_shard1_replica_n2] Schema name=minimal
[junit4] 2> 131134 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 131134 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n2' using configuration from collection collection1, trusted=true
[junit4] 2> 131134 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_36772.solr.core.collection1.shard1.replica_n2' (registry 'solr.core.collection1.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 131135 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 131135 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore [[collection1_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/tempDir-001/node3/collection1_shard1_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/tempDir-001/node3/./collection1_shard1_replica_n2/data/]
[junit4] 2> 131249 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 131379 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 131452 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=minimal
[junit4] 2> 131455 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 131455 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from collection collection1, trusted=true
[junit4] 2> 131488 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45394.solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@4731c9c8
[junit4] 2> 131489 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 131489 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/tempDir-001/node2/collection1_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_95AAE7548AC99BF-001/tempDir-001/node2/./collection1_shard1_replica_n1/data/]
[junit4] 2> 131672 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 131672 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 131673 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 131673 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 131696 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7d0077a2[collection1_shard1_replica_n2] main]
[junit4] 2> 131713 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 131713 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 131714 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 131761 INFO (searcherExecutor-164-thread-1-processing-n:127.0.0.1:36772_solr x:collection1_shard1_replica_n2 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore [collection1_shard1_replica_n2] Registered new searcher Searcher@7d0077a2[collection1_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 131762 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593133993498247168
[junit4] 2> 131862 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 131985 INFO (zkCallback-77-thread-2-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 132001 INFO (zkCallback-98-thread-2-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 132001 INFO (zkCallback-98-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 132001 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 132001 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 132002 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 132002 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 132004 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4637b0ec[collection1_shard1_replica_n1] main]
[junit4] 2> 132018 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 132018 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 132019 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 132019 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593133993767731200
[junit4] 2> 132135 INFO (searcherExecutor-165-thread-1-processing-n:127.0.0.1:45394_solr x:collection1_shard1_replica_n1 s:shard1 c:collection1 r:core_node3) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [collection1_shard1_replica_n1] Registered new searcher Searcher@4637b0ec[collection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 132250 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 132367 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 132367 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 132367 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36772/solr/collection1_shard1_replica_n2/
[junit4] 2> 132372 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n2 url=http://127.0.0.1:36772/solr START replicas=[http://127.0.0.1:45394/solr/collection1_shard1_replica_n1/] nUpdates=100
[junit4] 2> 132430 INFO (qtp131808664-276) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.c.S.Request [collection1_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=34
[junit4] 2> 132679 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n2 url=http://127.0.0.1:36772/solr DONE. We have no versions. sync failed.
[junit4] 2> 132679 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 132679 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 132679 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 132733 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36772/solr/collection1_shard1_replica_n2/ shard1
[junit4] 2> 132862 INFO (zkCallback-77-thread-2-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 132869 INFO (zkCallback-98-thread-2-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 132869 INFO (zkCallback-98-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 132906 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 132922 INFO (qtp1365257422-341) [n:127.0.0.1:36772_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n2&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2849
[junit4] 2> 133027 INFO (zkCallback-98-thread-2-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 133027 INFO (zkCallback-77-thread-2-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 133027 INFO (zkCallback-98-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 133193 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=1&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3153
[junit4] 2> 133237 INFO (qtp1365257422-337) [n:127.0.0.1:36772_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 133251 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 133304 INFO (zkCallback-77-thread-2-processing-n:127.0.0.1:45394_solr) [n:127.0.0.1:45394_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 133307 INFO (zkCallback-98-thread-2-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 133315 INFO (zkCallback-98-thread-1-processing-n:127.0.0.1:36772_solr) [n:127.0.0.1:36772_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [5])
[junit4] 2> 134250 INFO (qtp1365257422-337) [n:127.0.0.1:36772_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&name=collection1&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=5497
[junit4] 2> 134266 DEBUG (ScheduledTrigger-137-thread-1) [n:127.0.0.1:45394_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 5
[junit4] 2> 134525 INFO (qtp102416150-390) [n:127.0.0.1:44881_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=48
[junit4] 2> 134542 INFO (qtp131808664-274) [n:127.0.0.1:45394_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={wt=javabin&version=2&key=solr.core.collection1.shard1.replica_n1:INDEX.sizeInBytes} status=0 QTime=0
[junit4] 2> 134568 INFO (qtp131808664-260) [n:127.0.0.1:45394_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=15
[junit4] 2> 134
[...truncated too long message...]
common.cloud.SolrZkClient.create(SolrZkClient.java:365)
[junit4] 2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:885)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:704)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:703)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$900(ZkStateReader.java:72)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1090)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1085)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:269)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 90493 INFO (jetty-closer-8-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6ba98f29{/solr,null,UNAVAILABLE}
[junit4] 2> 90493 INFO (jetty-closer-8-thread-3) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 90598 INFO (jetty-closer-8-thread-4) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@17927b3a{/solr,null,UNAVAILABLE}
[junit4] 2> 90598 INFO (jetty-closer-8-thread-4) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 119265 WARN (jetty-closer-8-thread-1) [ ] o.e.j.u.t.QueuedThreadPool QueuedThreadPool@qtp1429719128{STOPPING,8<=8<=10000,i=0,q=1} Couldn't stop Thread[qtp1429719128-93,5,TGRP-TestV2Request]
[junit4] 2> 119266 ERROR (SUITE-TestV2Request-seed#[1A3FDF5A1DEC7E28]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 119270 INFO (SUITE-TestV2Request-seed#[1A3FDF5A1DEC7E28]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46868 46868
[junit4] 2> 119572 INFO (Thread-4) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46868 46868
[junit4] 2> 119574 WARN (Thread-4) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 13 /solr/collections/test/terms/shard2
[junit4] 2> 13 /solr/collections/test/terms/shard1
[junit4] 2> 12 /solr/configs/config
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 4 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 70 /solr/collections/test/state.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> 119574 INFO (SUITE-TestV2Request-seed#[1A3FDF5A1DEC7E28]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 22, 2018 9:45:31 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 2 leaked thread(s).
[junit4] 2> Feb 22, 2018 9:45:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.request.TestV2Request:
[junit4] 2> 1) Thread[id=93, name=qtp1429719128-93, state=TIMED_WAITING, group=TGRP-TestV2Request]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 22, 2018 9:45:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=93, name=qtp1429719128-93, state=TIMED_WAITING, group=TGRP-TestV2Request]
[junit4] 2> Feb 22, 2018 9:45:54 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=93, name=qtp1429719128-93, state=TIMED_WAITING, group=TGRP-TestV2Request]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1013, maxMBSortInHeap=7.308226226321597, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@53bff983), locale=hi, timezone=Africa/Mogadishu
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=2,free=232006560,total=324009984
[junit4] 2> NOTE: All tests run in this JVM: [SolrDocumentTest, SubtractEvaluatorTest, ClientUtilsTest, AbsoluteValueEvaluatorTest, MergeIndexesEmbeddedTest, TestV2Request]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestV2Request -Dtests.seed=1A3FDF5A1DEC7E28 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=hi -Dtests.timezone=Africa/Mogadishu -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestV2Request (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.request.TestV2Request:
[junit4] > 1) Thread[id=93, name=qtp1429719128-93, state=TIMED_WAITING, group=TGRP-TestV2Request]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([1A3FDF5A1DEC7E28]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=93, name=qtp1429719128-93, state=TIMED_WAITING, group=TGRP-TestV2Request]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([1A3FDF5A1DEC7E28]:0)
[junit4] Completed [29/146 (1!)] on J1 in 112.91s, 3 tests, 2 errors <<< FAILURES!
[...truncated 49499 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2359 - Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2359/
1 tests failed.
FAILED: org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataV1
Error Message:
Unexpected status: HTTP/1.1 400 Bad Request
Stack Trace:
java.lang.AssertionError: Unexpected status: HTTP/1.1 400 Bad Request
at __randomizedtesting.SeedInfo.seed([F425BB69EAAB9BAB:254505A0CA0788F2]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.AliasIntegrationTest.assertSuccess(AliasIntegrationTest.java:315)
at org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataV1(AliasIntegrationTest.java:250)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12853 lines...]
[junit4] Suite: org.apache.solr.cloud.AliasIntegrationTest
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/init-core-data-001
[junit4] 2> 850423 WARN (SUITE-AliasIntegrationTest-seed#[F425BB69EAAB9BAB]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 850423 INFO (SUITE-AliasIntegrationTest-seed#[F425BB69EAAB9BAB]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 850425 INFO (SUITE-AliasIntegrationTest-seed#[F425BB69EAAB9BAB]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 850425 INFO (SUITE-AliasIntegrationTest-seed#[F425BB69EAAB9BAB]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 850426 INFO (SUITE-AliasIntegrationTest-seed#[F425BB69EAAB9BAB]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001
[junit4] 2> 850426 INFO (SUITE-AliasIntegrationTest-seed#[F425BB69EAAB9BAB]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 850426 INFO (Thread-3095) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 850426 INFO (Thread-3095) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 850428 ERROR (Thread-3095) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 850526 INFO (SUITE-AliasIntegrationTest-seed#[F425BB69EAAB9BAB]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:41231
[junit4] 2> 850528 INFO (zkConnectionManagerCallback-1835-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 850532 INFO (jetty-launcher-1832-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 850532 INFO (jetty-launcher-1832-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T23:27:37+02:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 850533 INFO (jetty-launcher-1832-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 850533 INFO (jetty-launcher-1832-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 850533 INFO (jetty-launcher-1832-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 850534 INFO (jetty-launcher-1832-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 850534 INFO (jetty-launcher-1832-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 850534 INFO (jetty-launcher-1832-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 850534 INFO (jetty-launcher-1832-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@430ad05{/solr,null,AVAILABLE}
[junit4] 2> 850534 INFO (jetty-launcher-1832-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4aa7cb08{/solr,null,AVAILABLE}
[junit4] 2> 850534 INFO (jetty-launcher-1832-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5630bb3b{SSL,[ssl, http/1.1]}{127.0.0.1:32957}
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6dd42ccb{SSL,[ssl, http/1.1]}{127.0.0.1:52641}
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-1) [ ] o.e.j.s.Server Started @853034ms
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-2) [ ] o.e.j.s.Server Started @853034ms
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=32957}
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=52641}
[junit4] 2> 850535 ERROR (jetty-launcher-1832-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 850535 ERROR (jetty-launcher-1832-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T17:01:04.927Z
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 850535 INFO (jetty-launcher-1832-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T17:01:04.927Z
[junit4] 2> 850536 INFO (zkConnectionManagerCallback-1837-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 850537 INFO (zkConnectionManagerCallback-1839-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 850537 INFO (jetty-launcher-1832-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 850537 INFO (jetty-launcher-1832-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 850550 INFO (jetty-launcher-1832-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41231/solr
[junit4] 2> 850551 INFO (jetty-launcher-1832-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:41231/solr
[junit4] 2> 850553 INFO (zkConnectionManagerCallback-1847-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 850553 INFO (zkConnectionManagerCallback-1845-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 850556 INFO (zkConnectionManagerCallback-1849-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 850557 INFO (zkConnectionManagerCallback-1851-thread-1-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 850597 INFO (jetty-launcher-1832-thread-1) [n:127.0.0.1:32957_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 850598 INFO (jetty-launcher-1832-thread-1) [n:127.0.0.1:32957_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:32957_solr
[junit4] 2> 850600 INFO (jetty-launcher-1832-thread-1) [n:127.0.0.1:32957_solr ] o.a.s.c.Overseer Overseer (id=73340184154210310-127.0.0.1:32957_solr-n_0000000000) starting
[junit4] 2> 850619 INFO (jetty-launcher-1832-thread-1) [n:127.0.0.1:32957_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:32957_solr
[junit4] 2> 850625 INFO (zkCallback-1850-thread-1-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 850654 INFO (jetty-launcher-1832-thread-2) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 850656 INFO (jetty-launcher-1832-thread-2) [n:127.0.0.1:52641_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 850658 INFO (jetty-launcher-1832-thread-2) [n:127.0.0.1:52641_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:52641_solr
[junit4] 2> 850668 INFO (zkCallback-1850-thread-1-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 850669 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 850715 INFO (jetty-launcher-1832-thread-2) [n:127.0.0.1:52641_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52641.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 850716 INFO (jetty-launcher-1832-thread-1) [n:127.0.0.1:32957_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32957.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 850729 INFO (jetty-launcher-1832-thread-2) [n:127.0.0.1:52641_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52641.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 850729 INFO (jetty-launcher-1832-thread-2) [n:127.0.0.1:52641_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52641.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 850730 INFO (jetty-launcher-1832-thread-1) [n:127.0.0.1:32957_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32957.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 850730 INFO (jetty-launcher-1832-thread-1) [n:127.0.0.1:32957_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32957.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 850730 INFO (jetty-launcher-1832-thread-2) [n:127.0.0.1:52641_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node2/.
[junit4] 2> 850731 INFO (jetty-launcher-1832-thread-1) [n:127.0.0.1:32957_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node1/.
[junit4] 2> 850772 INFO (zkConnectionManagerCallback-1857-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 850775 INFO (zkConnectionManagerCallback-1861-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 850776 INFO (SUITE-AliasIntegrationTest-seed#[F425BB69EAAB9BAB]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 850777 INFO (SUITE-AliasIntegrationTest-seed#[F425BB69EAAB9BAB]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:41231/solr ready
[junit4] 2> 850805 INFO (TEST-AliasIntegrationTest.testModifyMetadataV1-seed#[F425BB69EAAB9BAB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testModifyMetadataV1
[junit4] 2> 850848 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterstatus with params action=CLUSTERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 850848 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=CLUSTERSTATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 850877 INFO (qtp1615471457-10186) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection1meta&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 850879 INFO (OverseerThreadFactory-3715-thread-1-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1meta
[junit4] 2> 850988 INFO (OverseerStateUpdate-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1meta",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1meta_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:32957/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 850990 INFO (OverseerStateUpdate-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1meta",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1meta_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:52641/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 851208 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1meta_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1meta&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 851208 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 851209 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1meta_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1meta&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 851209 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 851314 INFO (zkCallback-1850-thread-1-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 851314 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 851314 INFO (zkCallback-1850-thread-2-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 851314 INFO (zkCallback-1848-thread-2-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852221 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 852221 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 852227 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.IndexSchema [collection1meta_shard1_replica_n1] Schema name=minimal
[junit4] 2> 852228 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.IndexSchema [collection1meta_shard2_replica_n2] Schema name=minimal
[junit4] 2> 852230 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 852230 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1meta_shard1_replica_n1' using configuration from collection collection1meta, trusted=true
[junit4] 2> 852239 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32957.solr.core.collection1meta.shard1.replica_n1' (registry 'solr.core.collection1meta.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 852239 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 852239 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1meta_shard2_replica_n2' using configuration from collection collection1meta, trusted=true
[junit4] 2> 852239 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 852239 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrCore [[collection1meta_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node1/collection1meta_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node1/./collection1meta_shard1_replica_n1/data/]
[junit4] 2> 852239 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52641.solr.core.collection1meta.shard2.replica_n2' (registry 'solr.core.collection1meta.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 852239 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 852239 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrCore [[collection1meta_shard2_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node2/collection1meta_shard2_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node2/./collection1meta_shard2_replica_n2/data/]
[junit4] 2> 852282 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 852282 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 852284 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 852284 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 852285 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@af35f7d[collection1meta_shard1_replica_n1] main]
[junit4] 2> 852286 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 852286 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 852287 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 852287 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 852287 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 852287 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593121299946799104
[junit4] 2> 852288 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 852288 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 852288 INFO (searcherExecutor-3720-thread-1-processing-n:127.0.0.1:32957_solr x:collection1meta_shard1_replica_n1 s:shard1 c:collection1meta r:core_node3) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrCore [collection1meta_shard1_replica_n1] Registered new searcher Searcher@af35f7d[collection1meta_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 852291 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@1b1276de[collection1meta_shard2_replica_n2] main]
[junit4] 2> 852292 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 852292 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 852293 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 852293 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593121299953090560
[junit4] 2> 852296 INFO (searcherExecutor-3721-thread-1-processing-n:127.0.0.1:52641_solr x:collection1meta_shard2_replica_n2 s:shard2 c:collection1meta r:core_node4) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrCore [collection1meta_shard2_replica_n2] Registered new searcher Searcher@1b1276de[collection1meta_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 852301 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 852301 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 852302 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:32957/solr/collection1meta_shard1_replica_n1/
[junit4] 2> 852302 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 852302 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:32957/solr/collection1meta_shard1_replica_n1/ has no replicas
[junit4] 2> 852302 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 852307 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:32957/solr/collection1meta_shard1_replica_n1/ shard1
[junit4] 2> 852308 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 852308 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 852308 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:52641/solr/collection1meta_shard2_replica_n2/
[junit4] 2> 852308 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 852308 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:52641/solr/collection1meta_shard2_replica_n2/ has no replicas
[junit4] 2> 852308 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 852311 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:52641/solr/collection1meta_shard2_replica_n2/ shard2
[junit4] 2> 852412 INFO (zkCallback-1850-thread-2-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852412 INFO (zkCallback-1850-thread-1-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852413 INFO (zkCallback-1848-thread-2-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852413 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852459 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 852461 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1meta_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1meta&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1252
[junit4] 2> 852462 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 852463 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1meta_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1meta&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1255
[junit4] 2> 852469 INFO (qtp1615471457-10186) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 852564 INFO (zkCallback-1850-thread-1-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852564 INFO (zkCallback-1848-thread-2-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852564 INFO (zkCallback-1850-thread-2-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852565 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852884 INFO (OverseerCollectionConfigSetProcessor-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 853469 INFO (qtp1615471457-10186) [n:127.0.0.1:32957_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection1meta&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2591
[junit4] 2> 853470 INFO (qtp1615471457-10190) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection2meta&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 853472 INFO (OverseerThreadFactory-3715-thread-2-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection2meta
[junit4] 2> 853680 INFO (OverseerStateUpdate-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection2meta",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection2meta_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:52641/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 853881 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=collection2meta_shard1_replica_n1&action=CREATE&numShards=1&collection=collection2meta&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 853987 INFO (zkCallback-1848-thread-2-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 853987 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 854894 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 854900 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.s.IndexSchema [collection2meta_shard1_replica_n1] Schema name=minimal
[junit4] 2> 854902 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 854902 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection2meta_shard1_replica_n1' using configuration from collection collection2meta, trusted=true
[junit4] 2> 854903 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52641.solr.core.collection2meta.shard1.replica_n1' (registry 'solr.core.collection2meta.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 854903 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 854903 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SolrCore [[collection2meta_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node2/collection2meta_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node2/./collection2meta_shard1_replica_n1/data/]
[junit4] 2> 854957 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 854957 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 854958 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 854958 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 854959 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@786fe906[collection2meta_shard1_replica_n1] main]
[junit4] 2> 854960 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 854960 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 854961 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 854966 INFO (searcherExecutor-3730-thread-1-processing-n:127.0.0.1:52641_solr x:collection2meta_shard1_replica_n1 s:shard1 c:collection2meta r:core_node2) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SolrCore [collection2meta_shard1_replica_n1] Registered new searcher Searcher@786fe906[collection2meta_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 854966 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593121302755934208
[junit4] 2> 854972 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 854972 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 854972 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:52641/solr/collection2meta_shard1_replica_n1/
[junit4] 2> 854972 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 854972 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:52641/solr/collection2meta_shard1_replica_n1/ has no replicas
[junit4] 2> 854972 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 854975 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:52641/solr/collection2meta_shard1_replica_n1/ shard1
[junit4] 2> 855077 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 855077 INFO (zkCallback-1848-thread-2-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 855126 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 855128 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=collection2meta_shard1_replica_n1&action=CREATE&numShards=1&collection=collection2meta&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1246
[junit4] 2> 855133 INFO (qtp1615471457-10190) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 855229 INFO (zkCallback-1848-thread-2-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 855229 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 855473 INFO (OverseerCollectionConfigSetProcessor-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 856134 INFO (qtp1615471457-10190) [n:127.0.0.1:32957_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection2meta&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2664
[junit4] 2> 856141 INFO (qtp1615471457-10186) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :modifyalias with params metadata.bar=bam&metadata.foo=baz&name=testModifyMetadataV1&action=MODIFYALIAS&wt=xml and sendToOCPQueue=true
[junit4] 2> 856147 ERROR (OverseerThreadFactory-3715-thread-3-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Collection: testModifyMetadataV1 operation: modifyalias failed:org.apache.solr.common.SolrException: Can't modify non-existent alias testModifyMetadataV1
[junit4] 2> at org.apache.solr.cloud.api.collections.ModifyAliasCmd.call(ModifyAliasCmd.java:57)
[junit4] 2> at org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:254)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:469)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 856149 INFO (qtp1615471457-10186) [n:127.0.0.1:32957_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={metadata.bar=bam&metadata.foo=baz&name=testModifyMetadataV1&action=MODIFYALIAS&wt=xml} status=400 QTime=7
[junit4] 2> <?xml version="1.0" encoding="UTF-8"?>
[junit4] 2> <response>
[junit4] 2>
[junit4] 2> <lst name="responseHeader">
[junit4] 2> <int name="status">400</int>
[junit4] 2> <int name="QTime">7</int>
[junit4] 2> </lst>
[junit4] 2> <str name="Operation modifyalias caused exception:">org.apache.solr.common.SolrException:org.apache.solr.common.SolrException: Can't modify non-existent alias testModifyMetadataV1</str>
[junit4] 2> <lst name="exception">
[junit4] 2> <str name="msg">Can't modify non-existent alias testModifyMetadataV1</str>
[junit4] 2> <int name="rspCode">400</int>
[junit4] 2> </lst>
[junit4] 2> <lst name="error">
[junit4] 2> <lst name="metadata">
[junit4] 2> <str name="error-class">org.apache.solr.common.SolrException</str>
[junit4] 2> <str name="root-error-class">org.apache.solr.common.SolrException</str>
[junit4] 2> </lst>
[junit4] 2> <str name="msg">Can't modify non-existent alias testModifyMetadataV1</str>
[junit4] 2> <int name="code">400</int>
[junit4] 2> </lst>
[junit4] 2> </response>
[junit4] 2>
[junit4] 2> 856150 INFO (TEST-AliasIntegrationTest.testModifyMetadataV1-seed#[F425BB69EAAB9BAB]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testModifyMetadataV1
[junit4] 2> 856152 INFO (qtp1615471457-10190) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :listaliases with params action=LISTALIASES&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 856152 INFO (qtp1615471457-10190) [n:127.0.0.1:32957_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=LISTALIASES&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 856153 INFO (qtp1615471457-10186) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :deletealias with params name=testModifyMetadataV1&action=DELETEALIAS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 856159 INFO (qtp1615471457-10186) [n:127.0.0.1:32957_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=testModifyMetadataV1&action=DELETEALIAS&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 856160 INFO (TEST-AliasIntegrationTest.testModifyMetadataV1-seed#[F425BB69EAAB9BAB]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 856161 INFO (qtp1615471457-10190) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=collection2meta&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 856166 INFO (OverseerCollectionConfigSetProcessor-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 856168 INFO (OverseerThreadFactory-3715-thread-5-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=null
[junit4] 2> 856169 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection2meta.shard1.replica_n1, tag=null
[junit4] 2> 856169 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@48067acd: rootName = solr_52641, domain = solr.core.collection2meta.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection2meta.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@69cf1d8b
[junit4] 2> 856180 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr ] o.a.s.c.SolrCore [collection2meta_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@615f7875
[junit4] 2> 856180 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection2meta.shard1.replica_n1, tag=1633646709
[junit4] 2> 856180 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection2meta.shard1.leader, tag=1633646709
[junit4] 2> 856183 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=collection2meta_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=14
[junit4] 2> 856892 INFO (qtp1615471457-10190) [n:127.0.0.1:32957_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=collection2meta&action=DELETE&wt=javabin&version=2} status=0 QTime=731
[junit4] 2> 856893 INFO (qtp1615471457-10186) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :delete with params name=collection1meta&action=DELETE&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 856894 INFO (OverseerThreadFactory-3715-thread-5-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Executing Collection Cmd=action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true, asyncId=null
[junit4] 2> 856895 INFO (OverseerCollectionConfigSetProcessor-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 856895 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1meta.shard2.replica_n2, tag=null
[junit4] 2> 856895 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@76640b8b: rootName = solr_52641, domain = solr.core.collection1meta.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.collection1meta.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@4d77122f
[junit4] 2> 856896 INFO (qtp1615471457-10179) [n:127.0.0.1:32957_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1meta.shard1.replica_n1, tag=null
[junit4] 2> 856896 INFO (qtp1615471457-10179) [n:127.0.0.1:32957_solr ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@71282f2f: rootName = solr_32957, domain = solr.core.collection1meta.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.collection1meta.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@1f1c3378
[junit4] 2> 856909 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr ] o.a.s.c.SolrCore [collection1meta_shard2_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@6c35d8f1
[junit4] 2> 856909 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1meta.shard2.replica_n2, tag=1815468273
[junit4] 2> 856909 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1meta.shard2.leader, tag=1815468273
[junit4] 2> 856911 INFO (qtp1615471457-10179) [n:127.0.0.1:32957_solr ] o.a.s.c.SolrCore [collection1meta_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@32779aa8
[junit4] 2> 856911 INFO (qtp1615471457-10179) [n:127.0.0.1:32957_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1meta.shard1.replica_n1, tag=846699176
[junit4] 2> 856911 INFO (qtp1615471457-10179) [n:127.0.0.1:32957_solr ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1meta.shard1.leader, tag=846699176
[junit4] 2> 856914 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=collection1meta_shard2_replica_n2&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 856914 INFO (qtp1615471457-10179) [n:127.0.0.1:32957_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={deleteInstanceDir=true&core=collection1meta_shard1_replica_n1&qt=/admin/cores&deleteDataDir=true&action=UNLOAD&wt=javabin&version=2} status=0 QTime=18
[junit4] 2> 857624 INFO (qtp1615471457-10186) [n:127.0.0.1:32957_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={name=collection1meta&action=DELETE&wt=javabin&version=2} status=0 QTime=731
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=AliasIntegrationTest -Dtests.method=testModifyMetadataV1 -Dtests.seed=F425BB69EAAB9BAB -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=mt -Dtests.timezone=America/Curacao -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 6.85s J2 | AliasIntegrationTest.testModifyMetadataV1 <<<
[junit4] > Throwable #1: java.lang.AssertionError: Unexpected status: HTTP/1.1 400 Bad Request
[junit4] > at __randomizedtesting.SeedInfo.seed([F425BB69EAAB9BAB:254505A0CA0788F2]:0)
[junit4] > at org.apache.solr.cloud.AliasIntegrationTest.assertSuccess(AliasIntegrationTest.java:315)
[junit4] > at org.apache.solr.cloud.AliasIntegrationTest.testModifyMetadataV1(AliasIntegrationTest.java:250)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 857646 INFO (TEST-AliasIntegrationTest.testModifyMetadataV2-seed#[F425BB69EAAB9BAB]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testModifyMetadataV2
[junit4] 2> 857648 INFO (qtp1615471457-10190) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection1meta&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 857649 INFO (OverseerThreadFactory-3715-thread-5-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection1meta
[junit4] 2> 857649 INFO (OverseerCollectionConfigSetProcessor-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000010 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 857857 INFO (OverseerStateUpdate-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1meta",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1meta_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:52641/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 857859 INFO (OverseerStateUpdate-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1meta",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1meta_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:32957/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 858061 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1meta_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1meta&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 858061 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1meta_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1meta&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 858167 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 858167 INFO (zkCallback-1848-thread-2-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 858167 INFO (zkCallback-1850-thread-2-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 858167 INFO (zkCallback-1850-thread-3-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 859076 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 859076 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 859112 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.IndexSchema [collection1meta_shard2_replica_n2] Schema name=minimal
[junit4] 2> 859112 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.IndexSchema [collection1meta_shard1_replica_n1] Schema name=minimal
[junit4] 2> 859114 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 859114 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1meta_shard2_replica_n2' using configuration from collection collection1meta, trusted=true
[junit4] 2> 859114 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 859114 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1meta_shard1_replica_n1' using configuration from collection collection1meta, trusted=true
[junit4] 2> 859115 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_32957.solr.core.collection1meta.shard2.replica_n2' (registry 'solr.core.collection1meta.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 859115 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52641.solr.core.collection1meta.shard1.replica_n1' (registry 'solr.core.collection1meta.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 859115 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 859115 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrCore [[collection1meta_shard2_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node1/collection1meta_shard2_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node1/./collection1meta_shard2_replica_n2/data/]
[junit4] 2> 859115 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 859115 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrCore [[collection1meta_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node2/collection1meta_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node2/./collection1meta_shard1_replica_n1/data/]
[junit4] 2> 859162 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 859162 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 859163 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 859163 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 859165 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@a12bfbb[collection1meta_shard1_replica_n1] main]
[junit4] 2> 859166 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 859166 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 859167 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 859168 INFO (searcherExecutor-3736-thread-1-processing-n:127.0.0.1:52641_solr x:collection1meta_shard1_replica_n1 s:shard1 c:collection1meta r:core_node3) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SolrCore [collection1meta_shard1_replica_n1] Registered new searcher Searcher@a12bfbb[collection1meta_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 859168 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593121307162050560
[junit4] 2> 859169 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 859169 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 859170 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 859170 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 859173 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@1a748564[collection1meta_shard2_replica_n2] main]
[junit4] 2> 859174 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 859174 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 859175 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 859175 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 859175 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 859175 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:52641/solr/collection1meta_shard1_replica_n1/
[junit4] 2> 859175 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593121307169390592
[junit4] 2> 859175 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 859175 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:52641/solr/collection1meta_shard1_replica_n1/ has no replicas
[junit4] 2> 859175 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 859177 INFO (searcherExecutor-3735-thread-1-processing-n:127.0.0.1:32957_solr x:collection1meta_shard2_replica_n2 s:shard2 c:collection1meta r:core_node4) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SolrCore [collection1meta_shard2_replica_n2] Registered new searcher Searcher@1a748564[collection1meta_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 859178 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:52641/solr/collection1meta_shard1_replica_n1/ shard1
[junit4] 2> 859185 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 859185 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 859186 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:32957/solr/collection1meta_shard2_replica_n2/
[junit4] 2> 859186 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 859186 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.SyncStrategy https://127.0.0.1:32957/solr/collection1meta_shard2_replica_n2/ has no replicas
[junit4] 2> 859186 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 859189 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:32957/solr/collection1meta_shard2_replica_n2/ shard2
[junit4] 2> 859290 INFO (zkCallback-1850-thread-2-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 859290 INFO (zkCallback-1850-thread-3-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 859291 INFO (zkCallback-1848-thread-2-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 859291 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 859329 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 859333 INFO (qtp90322282-10189) [n:127.0.0.1:52641_solr c:collection1meta s:shard1 r:core_node3 x:collection1meta_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1meta_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1meta&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1271
[junit4] 2> 859340 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 859344 INFO (qtp1615471457-10188) [n:127.0.0.1:32957_solr c:collection1meta s:shard2 r:core_node4 x:collection1meta_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1meta_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1meta&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1283
[junit4] 2> 859346 INFO (qtp1615471457-10190) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 859445 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 859445 INFO (zkCallback-1850-thread-3-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 859445 INFO (zkCallback-1848-thread-2-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 859445 INFO (zkCallback-1850-thread-1-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1meta/state.json] for collection [collection1meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 859656 INFO (OverseerCollectionConfigSetProcessor-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000012 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 860346 INFO (qtp1615471457-10190) [n:127.0.0.1:32957_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf&name=collection1meta&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=2698
[junit4] 2> 860347 INFO (qtp1615471457-10186) [n:127.0.0.1:32957_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection2meta&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 860350 INFO (OverseerThreadFactory-3715-thread-5-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection collection2meta
[junit4] 2> 860561 INFO (OverseerStateUpdate-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection2meta",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection2meta_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:52641/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 860762 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=collection2meta_shard1_replica_n1&action=CREATE&numShards=1&collection=collection2meta&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 860868 INFO (zkCallback-1848-thread-2-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 860868 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection2meta/state.json] for collection [collection2meta] has occurred - updating... (live nodes size: [2])
[junit4] 2> 861778 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 861792 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.s.IndexSchema [collection2meta_shard1_replica_n1] Schema name=minimal
[junit4] 2> 861798 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 861798 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection2meta_shard1_replica_n1' using configuration from collection collection2meta, trusted=true
[junit4] 2> 861798 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_52641.solr.core.collection2meta.shard1.replica_n1' (registry 'solr.core.collection2meta.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15747963
[junit4] 2> 861798 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 861798 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.c.SolrCore [[collection2meta_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node2/collection2meta_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001/tempDir-001/node2/./collection2meta_shard1_replica_n1/data/]
[junit4] 2> 861843 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 861843 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 861844 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 861844 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 861845 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@569f7593[collection2meta_shard1_replica_n1] main]
[junit4] 2> 861846 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 861847 INFO (qtp90322282-10183) [n:127.0.0.1:52641_solr c:collection2meta s:shard1 r:core_node2 x:collection2meta_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_mana
[...truncated too long message...]
apache.solr.metrics.reporters.SolrJmxReporter@4a6a8403: rootName = solr_32957, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5ee57f1c
[junit4] 2> 914595 INFO (jetty-closer-1833-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 914595 INFO (jetty-closer-1833-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@26d3c95c: rootName = solr_52641, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@6cf6e78f
[junit4] 2> 914599 INFO (jetty-closer-1833-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 914599 INFO (jetty-closer-1833-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@57f709be: rootName = solr_52641, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@735a81d4
[junit4] 2> 914600 INFO (jetty-closer-1833-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:52641_solr
[junit4] 2> 914600 INFO (jetty-closer-1833-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 914600 INFO (jetty-closer-1833-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@54d57100: rootName = solr_32957, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@6cf6e78f
[junit4] 2> 914601 INFO (jetty-closer-1833-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 914601 INFO (zkCallback-1850-thread-2-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 914602 INFO (zkCallback-1848-thread-1-processing-n:127.0.0.1:52641_solr) [n:127.0.0.1:52641_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 914606 INFO (jetty-closer-1833-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 914607 INFO (jetty-closer-1833-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@14be800d: rootName = solr_32957, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@735a81d4
[junit4] 2> 914609 INFO (jetty-closer-1833-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@430ad05{/solr,null,UNAVAILABLE}
[junit4] 2> 914609 INFO (jetty-closer-1833-thread-2) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 914611 INFO (jetty-closer-1833-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:32957_solr
[junit4] 2> 914612 INFO (jetty-closer-1833-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 914612 INFO (zkCallback-1850-thread-2-processing-n:127.0.0.1:32957_solr) [n:127.0.0.1:32957_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 914613 INFO (jetty-closer-1833-thread-1) [ ] o.a.s.c.Overseer Overseer (id=73340184154210310-127.0.0.1:32957_solr-n_0000000000) closing
[junit4] 2> 914613 INFO (OverseerStateUpdate-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:32957_solr
[junit4] 2> 914614 WARN (OverseerAutoScalingTriggerThread-73340184154210310-127.0.0.1:32957_solr-n_0000000000) [n:127.0.0.1:32957_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 914619 INFO (jetty-closer-1833-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4aa7cb08{/solr,null,UNAVAILABLE}
[junit4] 2> 914619 INFO (jetty-closer-1833-thread-1) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 914622 ERROR (SUITE-AliasIntegrationTest-seed#[F425BB69EAAB9BAB]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 914623 INFO (SUITE-AliasIntegrationTest-seed#[F425BB69EAAB9BAB]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41231 41231
[junit4] 2> 914723 INFO (Thread-3095) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:41231 41231
[junit4] 2> 914724 WARN (Thread-3095) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 18 /solr/aliases.json
[junit4] 2> 16 /solr/configs/conf
[junit4] 2> 9 /solr/collections/collection2meta/terms/shard1
[junit4] 2> 9 /solr/collections/collection1meta/terms/shard2
[junit4] 2> 8 /solr/collections/collection1meta/terms/shard1
[junit4] 2> 3 /solr/collections/collection_two/terms/shard1
[junit4] 2> 3 /solr/collections/collection_old/terms/shard2
[junit4] 2> 3 /solr/collections/collection_old/terms/shard1
[junit4] 2> 3 /solr/collections/testErrorChecks-collection/terms/shard2
[junit4] 2> 3 /solr/collections/collection1/terms/shard2
[junit4] 2> 2 /solr/collections/collection_new/terms/shard1
[junit4] 2> 2 /solr/collections/collection2/terms/shard1
[junit4] 2> 2 /solr/collections/collection_one/terms/shard1
[junit4] 2> 2 /solr/collections/collection_one/terms/shard2
[junit4] 2> 2 /solr/collections/testErrorChecks-collection/terms/shard1
[junit4] 2> 2 /solr/security.json
[junit4] 2> 2 /solr/collections/collection1/terms/shard1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 257 /solr/aliases.json
[junit4] 2> 44 /solr/collections/collection1meta/state.json
[junit4] 2> 24 /solr/collections/collection2meta/state.json
[junit4] 2> 18 /solr/clusterprops.json
[junit4] 2> 18 /solr/clusterstate.json
[junit4] 2> 11 /solr/collections/collection_one/state.json
[junit4] 2> 11 /solr/collections/collection1/state.json
[junit4] 2> 11 /solr/collections/collection_old/state.json
[junit4] 2> 11 /solr/collections/testErrorChecks-collection/state.json
[junit4] 2> 6 /solr/collections/collection_new/state.json
[junit4] 2> 6 /solr/collections/collection_two/state.json
[junit4] 2> 6 /solr/collections/collection2/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 197 /solr/collections
[junit4] 2> 18 /solr/live_nodes
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.AliasIntegrationTest_F425BB69EAAB9BAB-001
[junit4] 2> Feb 22, 2018 5:02:09 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {a_t=PostingsFormat(name=LuceneFixedGap), id=FST50}, docValues:{}, maxPointsInLeafNode=326, maxMBSortInHeap=5.6601510636058805, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@533ffc22), locale=mt, timezone=America/Curacao
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=227613632,total=509083648
[junit4] 2> NOTE: All tests run in this JVM: [TestNoOpRegenerator, TestNamedUpdateProcessors, DateMathParserTest, ReplaceNodeNoTargetTest, TestCoreContainer, TestLazyCores, SpellCheckCollatorWithCollapseTest, TestMinMaxOnMultiValuedField, TestLFUCache, DistributedSpellCheckComponentTest, TestRealTimeGet, AnalyticsQueryTest, HdfsDirectoryTest, TestCloudSearcherWarming, PreAnalyzedFieldTest, HttpPartitionTest, StatsReloadRaceTest, BitVectorTest, DistribCursorPagingTest, TestDistributedStatsComponentCardinality, DistributedQueueTest, TestConfigsApi, DistributedFacetSimpleRefinementLongTailTest, CdcrRequestHandlerTest, TestRestManager, PingRequestHandlerTest, TestJsonRequest, BlockCacheTest, DistributedFacetPivotSmallTest, CdcrUpdateLogTest, PeerSyncReplicationTest, TestMiniSolrCloudClusterSSL, AutoAddReplicasIntegrationTest, HdfsChaosMonkeySafeLeaderTest, ClusterStateUpdateTest, TestChildDocTransformer, DeleteInactiveReplicaTest, TestHdfsUpdateLog, LargeFieldTest, SchemaApiFailureTest, LeaderElectionTest, ResponseLogComponentTest, DistributedFacetPivotLargeTest, TestTolerantUpdateProcessorCloud, PKIAuthenticationIntegrationTest, CollectionReloadTest, CoreMergeIndexesAdminHandlerTest, TestPivotHelperCode, OverseerRolesTest, MultiThreadedOCPTest, MetricUtilsTest, TestCloudDeleteByQuery, TestSolrDeletionPolicy2, TestDistributedMissingSort, CoreAdminHandlerTest, TestUpdate, ConnectionReuseTest, TestSizeLimitedDistributedMap, DocValuesTest, JvmMetricsTest, TestCloudPivotFacet, TestMaxScoreQueryParser, LukeRequestHandlerTest, SolrJmxReporterTest, TestInfoStreamLogging, HttpTriggerListenerTest, NumericFieldsTest, TestGenericDistributedQueue, TestCoreDiscovery, TermVectorComponentTest, TestTolerantSearch, TestHashQParserPlugin, DistributedQueryComponentOptimizationTest, TestDFRSimilarityFactory, TestSolrCloudWithKerberosAlt, AliasIntegrationTest]
[junit4] Completed [249/778 (1!)] on J2 in 65.07s, 8 tests, 1 failure <<< FAILURES!
[...truncated 51816 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2358 - Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2358/
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger
Error Message:
expected:<2> but was:<1>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<1>
at __randomizedtesting.SeedInfo.seed([E8165FBDA1B87F0A:7B0D17CFFF45243E]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger(TriggerIntegrationTest.java:1679)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 14256 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest
[junit4] 2> 2367037 INFO (SUITE-TriggerIntegrationTest-seed#[E8165FBDA1B87F0A]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_E8165FBDA1B87F0A-001/init-core-data-001
[junit4] 2> 2367038 WARN (SUITE-TriggerIntegrationTest-seed#[E8165FBDA1B87F0A]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=25 numCloses=25
[junit4] 2> 2367038 INFO (SUITE-TriggerIntegrationTest-seed#[E8165FBDA1B87F0A]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2367039 INFO (SUITE-TriggerIntegrationTest-seed#[E8165FBDA1B87F0A]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2367040 INFO (SUITE-TriggerIntegrationTest-seed#[E8165FBDA1B87F0A]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_E8165FBDA1B87F0A-001/tempDir-001
[junit4] 2> 2367040 INFO (SUITE-TriggerIntegrationTest-seed#[E8165FBDA1B87F0A]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2367040 INFO (Thread-3337) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2367040 INFO (Thread-3337) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2367046 ERROR (Thread-3337) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2367140 INFO (SUITE-TriggerIntegrationTest-seed#[E8165FBDA1B87F0A]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46604
[junit4] 2> 2367149 INFO (zkConnectionManagerCallback-6137-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2367159 INFO (jetty-launcher-6134-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T13:27:37-08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2367159 INFO (jetty-launcher-6134-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T13:27:37-08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2367161 INFO (jetty-launcher-6134-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2367161 INFO (jetty-launcher-6134-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2367161 INFO (jetty-launcher-6134-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2367161 INFO (jetty-launcher-6134-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2367161 INFO (jetty-launcher-6134-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2367161 INFO (jetty-launcher-6134-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2367161 INFO (jetty-launcher-6134-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@21dc8120{/solr,null,AVAILABLE}
[junit4] 2> 2367161 INFO (jetty-launcher-6134-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@431acaeb{/solr,null,AVAILABLE}
[junit4] 2> 2367161 INFO (jetty-launcher-6134-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@49e1603a{HTTP/1.1,[http/1.1]}{127.0.0.1:33725}
[junit4] 2> 2367162 INFO (jetty-launcher-6134-thread-1) [ ] o.e.j.s.Server Started @2369500ms
[junit4] 2> 2367162 INFO (jetty-launcher-6134-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=33725}
[junit4] 2> 2367162 INFO (jetty-launcher-6134-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@25dcdded{HTTP/1.1,[http/1.1]}{127.0.0.1:57572}
[junit4] 2> 2367162 ERROR (jetty-launcher-6134-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2367162 INFO (jetty-launcher-6134-thread-2) [ ] o.e.j.s.Server Started @2369501ms
[junit4] 2> 2367162 INFO (jetty-launcher-6134-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2367162 INFO (jetty-launcher-6134-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=57572}
[junit4] 2> 2367162 INFO (jetty-launcher-6134-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2367162 INFO (jetty-launcher-6134-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2367162 INFO (jetty-launcher-6134-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T13:48:20.200Z
[junit4] 2> 2367163 ERROR (jetty-launcher-6134-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2367163 INFO (jetty-launcher-6134-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2367163 INFO (jetty-launcher-6134-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2367163 INFO (jetty-launcher-6134-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2367163 INFO (jetty-launcher-6134-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T13:48:20.201Z
[junit4] 2> 2367165 INFO (zkConnectionManagerCallback-6139-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2367165 INFO (zkConnectionManagerCallback-6141-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2367165 INFO (jetty-launcher-6134-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2367167 INFO (jetty-launcher-6134-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2367179 INFO (jetty-launcher-6134-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46604/solr
[junit4] 2> 2367179 INFO (jetty-launcher-6134-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46604/solr
[junit4] 2> 2367183 INFO (zkConnectionManagerCallback-6147-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2367183 INFO (zkConnectionManagerCallback-6149-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2367189 INFO (zkConnectionManagerCallback-6151-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2367189 INFO (zkConnectionManagerCallback-6153-thread-1-processing-n:127.0.0.1:57572_solr) [n:127.0.0.1:57572_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2367273 INFO (jetty-launcher-6134-thread-2) [n:127.0.0.1:57572_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2367273 INFO (jetty-launcher-6134-thread-2) [n:127.0.0.1:57572_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:57572_solr
[junit4] 2> 2367274 INFO (jetty-launcher-6134-thread-2) [n:127.0.0.1:57572_solr ] o.a.s.c.Overseer Overseer (id=73339426247540742-127.0.0.1:57572_solr-n_0000000000) starting
[junit4] 2> 2367281 INFO (jetty-launcher-6134-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2367282 INFO (jetty-launcher-6134-thread-2) [n:127.0.0.1:57572_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57572_solr
[junit4] 2> 2367282 DEBUG (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 2367286 INFO (zkCallback-6152-thread-1-processing-n:127.0.0.1:57572_solr) [n:127.0.0.1:57572_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2367286 INFO (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2367288 INFO (jetty-launcher-6134-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33725_solr
[junit4] 2> 2367288 DEBUG (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 2367289 INFO (zkCallback-6152-thread-1-processing-n:127.0.0.1:57572_solr) [n:127.0.0.1:57572_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2367289 INFO (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2367294 DEBUG (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:33725_solr, 127.0.0.1:57572_solr]
[junit4] 2> 2367294 DEBUG (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 2367294 DEBUG (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 2367296 DEBUG (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2367296 DEBUG (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 2367296 DEBUG (ScheduledTrigger-6305-thread-1) [n:127.0.0.1:57572_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2367376 INFO (jetty-launcher-6134-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33725.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2367405 INFO (jetty-launcher-6134-thread-2) [n:127.0.0.1:57572_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57572.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2367409 INFO (jetty-launcher-6134-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33725.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2367409 INFO (jetty-launcher-6134-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_33725.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2367410 INFO (jetty-launcher-6134-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_E8165FBDA1B87F0A-001/tempDir-001/node1/.
[junit4] 2> 2367414 INFO (jetty-launcher-6134-thread-2) [n:127.0.0.1:57572_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57572.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2367414 INFO (jetty-launcher-6134-thread-2) [n:127.0.0.1:57572_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_57572.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2367415 INFO (jetty-launcher-6134-thread-2) [n:127.0.0.1:57572_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_E8165FBDA1B87F0A-001/tempDir-001/node2/.
[junit4] 2> 2367456 INFO (zkConnectionManagerCallback-6159-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2367462 INFO (zkConnectionManagerCallback-6163-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2367463 INFO (SUITE-TriggerIntegrationTest-seed#[E8165FBDA1B87F0A]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2367464 INFO (SUITE-TriggerIntegrationTest-seed#[E8165FBDA1B87F0A]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46604/solr ready
[junit4] 2> 2367545 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTriggerRestoreState
[junit4] 2> 2367546 INFO (qtp1984073611-25289) [n:127.0.0.1:33725_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2367554 INFO (qtp1984073611-25289) [n:127.0.0.1:33725_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=7
[junit4] 2> 2367558 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: .auto_add_replicas
[junit4] 2> 2367562 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 2
[junit4] 2> 2367562 DEBUG (zkCallback-6152-thread-1-processing-n:127.0.0.1:57572_solr) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 2367562 DEBUG (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 2367562 DEBUG (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2367562 DEBUG (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2367563 DEBUG (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 2367563 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2367564 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@25dcdded{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2367567 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=514267840
[junit4] 2> 2367567 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2367567 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@59354d7d: rootName = solr_57572, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@31337274
[junit4] 2> 2367575 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2367575 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@275a6279: rootName = solr_57572, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5121bc1f
[junit4] 2> 2367580 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2367580 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@772bd76b: rootName = solr_57572, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@581757e9
[junit4] 2> 2367583 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:57572_solr
[junit4] 2> 2367584 INFO (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 2367584 INFO (zkCallback-6152-thread-1-processing-n:127.0.0.1:57572_solr) [n:127.0.0.1:57572_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 2367585 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 2367585 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2367585 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.Overseer Overseer (id=73339426247540742-127.0.0.1:57572_solr-n_0000000000) closing
[junit4] 2> 2367585 INFO (OverseerStateUpdate-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:57572_solr
[junit4] 2> 2367592 DEBUG (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 2367586 WARN (OverseerAutoScalingTriggerThread-73339426247540742-127.0.0.1:57572_solr-n_0000000000) [n:127.0.0.1:57572_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 2367593 INFO (zkCallback-6162-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 2367593 INFO (zkCallback-6162-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 2367593 INFO (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:33725_solr
[junit4] 2> 2367594 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@431acaeb{/solr,null,UNAVAILABLE}
[junit4] 2> 2367595 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2367596 INFO (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.Overseer Overseer (id=73339426247540741-127.0.0.1:33725_solr-n_0000000001) starting
[junit4] 2> 2367601 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 2367602 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 2367602 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:33725_solr]
[junit4] 2> 2367603 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion -1
[junit4] 2> 2367603 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 2367604 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2367605 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 2367605 DEBUG (ScheduledTrigger-6314-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 2368606 DEBUG (ScheduledTrigger-6314-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 2369611 DEBUG (ScheduledTrigger-6314-thread-2) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 2370611 DEBUG (ScheduledTrigger-6314-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 2371611 DEBUG (ScheduledTrigger-6314-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 2372596 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T13:27:37-08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2372602 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2372602 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2372602 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2372602 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@421311dd{/solr,null,AVAILABLE}
[junit4] 2> 2372603 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5fe1c4fb{HTTP/1.1,[http/1.1]}{127.0.0.1:55720}
[junit4] 2> 2372603 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.Server Started @2374942ms
[junit4] 2> 2372603 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=55720}
[junit4] 2> 2372603 ERROR (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2372603 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2372603 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2372603 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2372604 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T13:48:25.642Z
[junit4] 2> 2372612 DEBUG (ScheduledTrigger-6314-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 1
[junit4] 2> 2372618 INFO (zkConnectionManagerCallback-6166-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2372619 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2372631 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46604/solr
[junit4] 2> 2372635 INFO (zkConnectionManagerCallback-6170-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2372642 INFO (zkConnectionManagerCallback-6172-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2372647 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:55720_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2372649 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:55720_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2372651 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:55720_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:55720_solr
[junit4] 2> 2372652 INFO (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2372652 INFO (zkCallback-6162-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2372652 INFO (zkCallback-6162-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2372658 INFO (zkCallback-6171-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2372709 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:55720_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55720.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2372719 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:55720_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55720.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2372719 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:55720_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_55720.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2372720 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:55720_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_E8165FBDA1B87F0A-001/tempDir-001/node3/.
[junit4] 2> 2372752 INFO (qtp1984073611-25298) [n:127.0.0.1:33725_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2372754 INFO (qtp1095183666-25360) [n:127.0.0.1:55720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2372754 DEBUG (qtp1984073611-25293) [n:127.0.0.1:33725_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2372755 INFO (qtp1984073611-25293) [n:127.0.0.1:33725_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 2372755 DEBUG (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 2372758 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T13:27:37-08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2372758 DEBUG (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:33725_solr, 127.0.0.1:55720_solr]
[junit4] 2> 2372758 DEBUG (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 2372758 DEBUG (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:33725_solr, 127.0.0.1:55720_solr]
[junit4] 2> 2372758 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 2372759 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2372760 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2372760 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2372760 INFO (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2372760 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4a813b61{/solr,null,AVAILABLE}
[junit4] 2> 2372760 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6d33aaad{HTTP/1.1,[http/1.1]}{127.0.0.1:44638}
[junit4] 2> 2372760 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.Server Started @2375099ms
[junit4] 2> 2372760 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44638}
[junit4] 2> 2372760 ERROR (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2372761 DEBUG (ScheduledTrigger-6314-thread-3) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 2372761 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2372761 DEBUG (ScheduledTrigger-6314-thread-3) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 2372761 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2372761 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2372761 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T13:48:25.799Z
[junit4] 2> 2372762 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 2372763 INFO (zkConnectionManagerCallback-6176-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2372763 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2372774 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46604/solr
[junit4] 2> 2372778 INFO (zkConnectionManagerCallback-6180-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2372785 INFO (zkConnectionManagerCallback-6182-thread-1-processing-n:127.0.0.1:44638_solr) [n:127.0.0.1:44638_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2372792 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:44638_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2372794 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:44638_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2372795 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:44638_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44638_solr
[junit4] 2> 2372796 INFO (zkCallback-6171-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2372796 INFO (zkCallback-6162-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2372796 INFO (zkCallback-6162-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2372797 INFO (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2372797 INFO (zkCallback-6181-thread-1-processing-n:127.0.0.1:44638_solr) [n:127.0.0.1:44638_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2372907 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:44638_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44638.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2372917 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:44638_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44638.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2372918 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:44638_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44638.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2372919 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:44638_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_E8165FBDA1B87F0A-001/tempDir-001/node4/.
[junit4] 2> 2373612 DEBUG (ScheduledTrigger-6314-thread-2) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2373765 DEBUG (ScheduledTrigger-6314-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 2373765 DEBUG (ScheduledTrigger-6314-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2373765 DEBUG (ScheduledTrigger-6314-thread-1) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:44638_solr at time 19559214023352613
[junit4] 2> 2374474 INFO (qtp1984073611-25293) [n:127.0.0.1:33725_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2374475 INFO (qtp581229017-25389) [n:127.0.0.1:44638_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2374477 INFO (qtp1095183666-25364) [n:127.0.0.1:55720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2374477 DEBUG (qtp581229017-25386) [n:127.0.0.1:44638_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2374478 INFO (qtp581229017-25386) [n:127.0.0.1:44638_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/cluster/autoscaling params={wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 2374478 DEBUG (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 2374479 DEBUG (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:33725_solr, 127.0.0.1:55720_solr, 127.0.0.1:44638_solr]
[junit4] 2> 2374479 DEBUG (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 2374479 DEBUG (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:33725_solr, 127.0.0.1:55720_solr, 127.0.0.1:44638_solr]
[junit4] 2> 2374479 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 2374480 INFO (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2374480 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:44638_solr
[junit4] 2> 2374481 DEBUG (ScheduledTrigger-6314-thread-3) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 2374481 DEBUG (ScheduledTrigger-6314-thread-3) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2374481 DEBUG (ScheduledTrigger-6314-thread-3) [n:127.0.0.1:33725_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger firing registered processor for nodes: [127.0.0.1:44638_solr] added at times [19559214738422965], now=19559214739189866
[junit4] 2> 2374481 DEBUG (ScheduledTrigger-6314-thread-3) [n:127.0.0.1:33725_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"457d008e91bcb5T9o7ewqbvr79nc97ac8rntpb7l",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":19559214738422965,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[19559214738422965],
[junit4] 2> "nodeNames":["127.0.0.1:44638_solr"]}}
[junit4] 2> 2374484 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 2374484 DEBUG (ScheduledTrigger-6314-thread-3) [n:127.0.0.1:33725_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_restore_trigger]
[junit4] 2> 2374484 INFO (ScheduledTrigger-6314-thread-3) [n:127.0.0.1:33725_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2374485 DEBUG (AutoscalingActionExecutor-6315-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"457d008e91bcb5T9o7ewqbvr79nc97ac8rntpb7l",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":19559214738422965,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[19559214738422965],
[junit4] 2> "_enqueue_time_":19559214742052832,
[junit4] 2> "nodeNames":["127.0.0.1:44638_solr"]}}
[junit4] 2> 2374485 INFO (TEST-TriggerIntegrationTest.testNodeAddedTriggerRestoreState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTriggerRestoreState
[junit4] 2> 2374525 DEBUG (AutoscalingActionExecutor-6315-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms
[junit4] 2> 2374527 DEBUG (AutoscalingActionExecutor-6315-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_restore_trigger after 5000ms
[junit4] 2> 2374528 DEBUG (AutoscalingActionExecutor-6315-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 43 ms for event id=457d008e91bcb5T9o7ewqbvr79nc97ac8rntpb7l
[junit4] 2> 2374582 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventFromRestoredState
[junit4] 2> 2374582 INFO (qtp1984073611-25297) [n:127.0.0.1:33725_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2374586 INFO (qtp1984073611-25297) [n:127.0.0.1:33725_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 2374586 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: .auto_add_replicas
[junit4] 2> 2374587 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.a.ScheduledTriggers -- removing trigger: node_added_restore_trigger
[junit4] 2> 2374599 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.a.TriggerIntegrationTest /autoscaling.json reset, new znode version 6
[junit4] 2> 2374599 DEBUG (zkCallback-6150-thread-1-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 2374599 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 2374600 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 2374600 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2374600 DEBUG (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 2374601 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2374603 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@49e1603a{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2374605 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1865012556
[junit4] 2> 2374605 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2374605 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@685011b: rootName = solr_33725, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@731919ce
[junit4] 2> 2374613 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2374613 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1011fe42: rootName = solr_33725, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5121bc1f
[junit4] 2> 2374618 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2374618 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1a7bd329: rootName = solr_33725, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@581757e9
[junit4] 2> 2374619 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:33725_solr
[junit4] 2> 2374620 INFO (zkCallback-6181-thread-1-processing-n:127.0.0.1:44638_solr) [n:127.0.0.1:44638_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2374620 INFO (zkCallback-6162-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2374620 INFO (zkCallback-6171-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2374620 INFO (zkCallback-6162-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2374621 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 2374621 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2374625 INFO (zkCallback-6162-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2374626 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.Overseer Overseer (id=73339426247540741-127.0.0.1:33725_solr-n_0000000001) closing
[junit4] 2> 2374626 INFO (OverseerStateUpdate-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:33725_solr
[junit4] 2> 2374626 INFO (zkCallback-6150-thread-2-processing-n:127.0.0.1:33725_solr) [n:127.0.0.1:33725_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 2374626 WARN (OverseerAutoScalingTriggerThread-73339426247540741-127.0.0.1:33725_solr-n_0000000001) [n:127.0.0.1:33725_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 2374626 DEBUG (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 2374627 INFO (zkCallback-6171-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:55720_solr
[junit4] 2> 2374628 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@21dc8120{/solr,null,UNAVAILABLE}
[junit4] 2> 2374630 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2374630 INFO (zkCallback-6171-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.Overseer Overseer (id=73339426247540747-127.0.0.1:55720_solr-n_0000000002) starting
[junit4] 2> 2374637 DEBUG (OverseerAutoScalingTriggerThread-73339426247540747-127.0.0.1:55720_solr-n_0000000002) [n:127.0.0.1:55720_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 2374638 DEBUG (OverseerAutoScalingTriggerThread-73339426247540747-127.0.0.1:55720_solr-n_0000000002) [n:127.0.0.1:55720_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 2374638 DEBUG (OverseerAutoScalingTriggerThread-73339426247540747-127.0.0.1:55720_solr-n_0000000002) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:55720_solr, 127.0.0.1:44638_solr]
[junit4] 2> 2374638 DEBUG (OverseerAutoScalingTriggerThread-73339426247540747-127.0.0.1:55720_solr-n_0000000002) [n:127.0.0.1:55720_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 2374638 DEBUG (OverseerAutoScalingTriggerThread-73339426247540747-127.0.0.1:55720_solr-n_0000000002) [n:127.0.0.1:55720_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 2374640 DEBUG (OverseerAutoScalingTriggerThread-73339426247540747-127.0.0.1:55720_solr-n_0000000002) [n:127.0.0.1:55720_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 2374640 DEBUG (OverseerAutoScalingTriggerThread-73339426247540747-127.0.0.1:55720_solr-n_0000000002) [n:127.0.0.1:55720_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 2374640 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2375641 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2376646 DEBUG (ScheduledTrigger-6331-thread-2) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2377646 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2378646 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2379643 INFO (qtp581229017-25390) [n:127.0.0.1:44638_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2379645 INFO (qtp1095183666-25359) [n:127.0.0.1:55720_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2379646 DEBUG (qtp581229017-25385) [n:127.0.0.1:44638_solr ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 2379646 INFO (qtp581229017-25385) [n:127.0.0.1:44638_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/autoscaling params={wt=javabin&version=2} status=0 QTime=9
[junit4] 2> 2379647 DEBUG (zkCallback-6171-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 2379647 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 2379649 DEBUG (zkCallback-6171-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:55720_solr, 127.0.0.1:44638_solr]
[junit4] 2> 2379649 DEBUG (zkCallback-6171-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEFRS instantiated with properties: {event=nodeAdded, waitFor=10, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.TriggerIntegrationTest$TestTriggerAction}], enabled=true}
[junit4] 2> 2379650 DEBUG (zkCallback-6171-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:55720_solr, 127.0.0.1:44638_solr]
[junit4] 2> 2379650 DEBUG (OverseerAutoScalingTriggerThread-73339426247540747-127.0.0.1:55720_solr-n_0000000002) [n:127.0.0.1:55720_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 2379652 INFO (OverseerAutoScalingTriggerThread-73339426247540747-127.0.0.1:55720_solr-n_0000000002) [n:127.0.0.1:55720_solr ] o.a.s.c.a.TriggerIntegrationTest TestTriggerAction init
[junit4] 2> 2379652 INFO (qtp581229017-25391) [n:127.0.0.1:44638_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2379653 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2379654 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 2379655 DEBUG (OverseerAutoScalingTriggerThread-73339426247540747-127.0.0.1:55720_solr-n_0000000002) [n:127.0.0.1:55720_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 2379658 INFO (qtp581229017-25391) [n:127.0.0.1:44638_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 2379659 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T13:27:37-08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2379660 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2379660 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2379660 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2379661 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@320fde3f{/solr,null,AVAILABLE}
[junit4] 2> 2379662 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@235118e7{HTTP/1.1,[http/1.1]}{127.0.0.1:46369}
[junit4] 2> 2379662 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.Server Started @2382001ms
[junit4] 2> 2379662 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46369}
[junit4] 2> 2379662 ERROR (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2379662 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2379663 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2379663 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2379663 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T13:48:32.701Z
[junit4] 2> 2379665 INFO (zkConnectionManagerCallback-6187-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379665 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2379676 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46604/solr
[junit4] 2> 2379677 INFO (zkConnectionManagerCallback-6191-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379683 INFO (zkConnectionManagerCallback-6193-thread-1-processing-n:127.0.0.1:46369_solr) [n:127.0.0.1:46369_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379692 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:46369_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2379694 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:46369_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2379696 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:46369_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46369_solr
[junit4] 2> 2379697 INFO (zkCallback-6162-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2379697 INFO (zkCallback-6162-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2379697 INFO (zkCallback-6162-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2379697 INFO (zkCallback-6181-thread-1-processing-n:127.0.0.1:44638_solr) [n:127.0.0.1:44638_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2379697 INFO (zkCallback-6171-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2379698 INFO (zkCallback-6192-thread-1-processing-n:127.0.0.1:46369_solr) [n:127.0.0.1:46369_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2379804 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:46369_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46369.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2379815 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:46369_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46369.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2379816 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:46369_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46369.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2379817 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:46369_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_E8165FBDA1B87F0A-001/tempDir-001/node5/.
[junit4] 2> 2380649 DEBUG (ScheduledTrigger-6331-thread-2) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2380659 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2380660 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2380660 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:46369_solr at time 19559220917900022
[junit4] 2> 2381650 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2381656 INFO (OverseerCollectionConfigSetProcessor-73339426247540747-127.0.0.1:55720_solr-n_0000000002) [n:127.0.0.1:55720_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2381661 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2381661 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2382651 DEBUG (ScheduledTrigger-6331-thread-2) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2382661 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2382662 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2383651 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2383662 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2383662 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2384651 DEBUG (ScheduledTrigger-6331-thread-2) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2384662 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2384662 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2385652 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2385663 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2385663 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2386652 DEBUG (ScheduledTrigger-6331-thread-2) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2386663 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2386663 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2387652 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2387663 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2387663 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2388653 DEBUG (ScheduledTrigger-6331-thread-2) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2388664 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2388664 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2389653 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2389664 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2389664 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2390653 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 2390664 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2390665 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 2390665 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_triggerEFRS firing registered processor for nodes: [127.0.0.1:46369_solr] added at times [19559220917900022], now=19559230922905383
[junit4] 2> 2390665 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"457d01fee512f6T9o7ewqbvr79nc97ac8rntpb7n",
[junit4] 2> "source":"node_added_triggerEFRS",
[junit4] 2> "eventTime":19559220917900022,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[19559220917900022],
[junit4] 2> "nodeNames":["127.0.0.1:46369_solr"]}}
[junit4] 2> 2390669 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_triggerEFRS]
[junit4] 2> 2390671 INFO (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 2390672 DEBUG (AutoscalingActionExecutor-6332-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"457d01fee512f6T9o7ewqbvr79nc97ac8rntpb7n",
[junit4] 2> "source":"node_added_triggerEFRS",
[junit4] 2> "eventTime":19559220917900022,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[19559220917900022],
[junit4] 2> "_enqueue_time_":19559230927268227,
[junit4] 2> "nodeNames":["127.0.0.1:46369_solr"]}}
[junit4] 2> 2390677 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T13:27:37-08:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2390678 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2390678 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2390678 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2390679 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@22a68bb{/solr,null,AVAILABLE}
[junit4] 2> 2390679 DEBUG (AutoscalingActionExecutor-6332-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 5000ms
[junit4] 2> 2390679 DEBUG (AutoscalingActionExecutor-6332-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_triggerEFRS after 5000ms
[junit4] 2> 2390679 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7a8e518{HTTP/1.1,[http/1.1]}{127.0.0.1:39500}
[junit4] 2> 2390680 DEBUG (AutoscalingActionExecutor-6332-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.a.ScheduledTriggers -- processing took 7 ms for event id=457d01fee512f6T9o7ewqbvr79nc97ac8rntpb7n
[junit4] 2> 2390680 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.Server Started @2393019ms
[junit4] 2> 2390680 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39500}
[junit4] 2> 2390681 ERROR (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2390681 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2390681 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2390681 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2390681 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T13:48:43.719Z
[junit4] 2> 2390684 INFO (zkConnectionManagerCallback-6197-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2390684 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2390696 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46604/solr
[junit4] 2> 2390697 INFO (zkConnectionManagerCallback-6201-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2390704 INFO (zkConnectionManagerCallback-6203-thread-1-processing-n:127.0.0.1:39500_solr) [n:127.0.0.1:39500_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2390714 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:39500_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2390717 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:39500_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2390719 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:39500_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39500_solr
[junit4] 2> 2390722 INFO (zkCallback-6192-thread-1-processing-n:127.0.0.1:46369_solr) [n:127.0.0.1:46369_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2390722 INFO (zkCallback-6162-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2390722 INFO (zkCallback-6162-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2390723 INFO (zkCallback-6202-thread-1-processing-n:127.0.0.1:39500_solr) [n:127.0.0.1:39500_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2390726 INFO (zkCallback-6162-thread-3) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2390726 INFO (zkCallback-6181-thread-1-processing-n:127.0.0.1:44638_solr) [n:127.0.0.1:44638_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2390727 INFO (zkCallback-6171-thread-1-processing-n:127.0.0.1:55720_solr) [n:127.0.0.1:55720_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2390802 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:39500_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39500.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2390813 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:39500_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39500.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2390813 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:39500_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39500.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@7b3284b7
[junit4] 2> 2390814 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [n:127.0.0.1:39500_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_E8165FBDA1B87F0A-001/tempDir-001/node6/.
[junit4] 2> 2395679 DEBUG (ScheduledTrigger-6331-thread-2) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2395680 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2395680 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2395680 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:39500_solr at time 19559235938115686
[junit4] 2> 2396681 DEBUG (ScheduledTrigger-6331-thread-2) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2396681 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2396681 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2397681 DEBUG (ScheduledTrigger-6331-thread-2) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2397681 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2397681 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2398681 DEBUG (ScheduledTrigger-6331-thread-2) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2398682 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2398682 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2399682 DEBUG (ScheduledTrigger-6331-thread-2) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2399682 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2399682 DEBUG (ScheduledTrigger-6331-thread-4) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2400682 DEBUG (ScheduledTrigger-6331-thread-1) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 2400682 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_triggerEFRS
[junit4] 2> 2400682 DEBUG (ScheduledTrigger-6331-thread-3) [n:127.0.0.1:55720_solr ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 2400875 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5fe1c4fb{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2400876 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=36431399
[junit4] 2> 2400876 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2400876 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@15d21c79: rootName = solr_55720, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@49a9c998
[junit4] 2> 2400884 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2400884 INFO (TEST-TriggerIntegrationTest.testEventFromRestoredState-seed#[E8165FBDA1B87F0A]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7ac573c7: rootName = solr_55720, domain = solr.jvm, service url = null, agent id =
[...truncated too long message...]
or.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2661711 INFO (jetty-closer-6135-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@65a8b459{/solr,null,UNAVAILABLE}
[junit4] 2> 2661712 INFO (jetty-closer-6135-thread-3) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 2661713 ERROR (SUITE-TriggerIntegrationTest-seed#[E8165FBDA1B87F0A]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2661714 INFO (SUITE-TriggerIntegrationTest-seed#[E8165FBDA1B87F0A]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46604 46604
[junit4] 2> 2661817 INFO (Thread-3337) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46604 46604
[junit4] 2> 2661822 WARN (Thread-3337) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 43 /solr/aliases.json
[junit4] 2> 43 /solr/clusterprops.json
[junit4] 2> 38 /solr/collections/collection1/terms/shard1
[junit4] 2> 27 /solr/security.json
[junit4] 2> 21 /solr/collections/testScheduledTrigger/terms/shard2
[junit4] 2> 15 /solr/collections/testScheduledTrigger/terms/shard1
[junit4] 2> 11 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 9 /solr/configs/conf
[junit4] 2> 7 /solr/configs/testScheduledTrigger.AUTOCREATED/managed-schema
[junit4] 2> 6 /solr/configs/testScheduledTrigger.AUTOCREATED
[junit4] 2> 3 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 65 /solr/collections/testScheduledTrigger/state.json
[junit4] 2> 54 /solr/collections/collection1/state.json
[junit4] 2> 43 /solr/clusterstate.json
[junit4] 2> 34 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 21 /solr/autoscaling.json
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73339426247540799-core_node4-n_0000000000
[junit4] 2> 2 /solr/collections/testScheduledTrigger/leader_elect/shard2/election/73339426247540783-core_node11-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/73339426247540805-127.0.0.1:41348_solr-n_0000000021
[junit4] 2> 2 /solr/overseer_elect/election/73339426247540814-127.0.0.1:36629_solr-n_0000000024
[junit4] 2> 2 /solr/collections/testScheduledTrigger/leader_elect/shard1/election/73339426247540774-core_node5-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 423 /solr/live_nodes
[junit4] 2> 110 /solr/collections
[junit4] 2> 21 /solr/overseer/queue-work
[junit4] 2> 21 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 21 /solr/overseer/collection-queue-work
[junit4] 2> 20 /solr/overseer/queue
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.TriggerIntegrationTest_E8165FBDA1B87F0A-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=Lucene50(blocksize=128), x_s=PostingsFormat(name=Memory)}, docValues:{}, maxPointsInLeafNode=671, maxMBSortInHeap=6.581489445821132, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@37a332ea), locale=en, timezone=Africa/Dakar
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=170494544,total=517472256
[junit4] 2> NOTE: All tests run in this JVM: [UUIDUpdateProcessorFallbackTest, TestSystemCollAutoCreate, DeleteShardTest, TestGroupingSearch, TestPerFieldSimilarityWithDefaultOverride, CursorPagingTest, TestCollationField, ExecutePlanActionTest, TestTrieFacet, RAMDirectoryFactoryTest, TestLegacyTerms, TestInfoStreamLogging, HdfsDirectoryFactoryTest, TestConfigSets, TestMacroExpander, SolrCoreCheckLockOnStartupTest, SpellCheckCollatorWithCollapseTest, TestImpersonationWithHadoopAuth, TestCSVLoader, TestQuerySenderNoQuery, BadCopyFieldTest, TestStressLiveNodes, BasicDistributedZkTest, HLLSerializationTest, TestFieldCacheSortRandom, TestExclusionRuleCollectionAccess, LeaderFailureAfterFreshStartTest, TestSolrCloudWithSecureImpersonation, DistributedQueryComponentOptimizationTest, HdfsChaosMonkeySafeLeaderTest, OpenCloseCoreStressTest, TestPHPSerializedResponseWriter, ZkStateReaderTest, RollingRestartTest, MetricTriggerTest, EnumFieldTest, ChaosMonkeySafeLeaderWithPullReplicasTest, RequestHandlersTest, TestSolrCoreSnapshots, AutoAddReplicasPlanActionTest, DocumentAnalysisRequestHandlerTest, TestSmileRequest, TestDistributedSearch, TestFieldSortValues, FieldAnalysisRequestHandlerTest, PluginInfoTest, TestNodeLostTrigger, HealthCheckHandlerTest, TestLuceneMatchVersion, AtomicUpdateProcessorFactoryTest, OpenExchangeRatesOrgProviderTest, TestSkipOverseerOperations, TestLRUStatsCache, UpdateLogTest, TestHighlightDedupGrouping, RuleEngineTest, TestIndexingPerformance, DirectUpdateHandlerTest, TestFileDictionaryLookup, TestHashQParserPlugin, IndexBasedSpellCheckerTest, MetricUtilsTest, TestCollationFieldDocValues, TestSQLHandlerNonCloud, SolrMetricManagerTest, StatsReloadRaceTest, CoreMergeIndexesAdminHandlerTest, TestSolrCLIRunExample, TestCoreBackup, AssignBackwardCompatibilityTest, TestNestedDocsSort, TestLegacyField, ConcurrentCreateRoutedAliasTest, EchoParamsTest, UninvertDocValuesMergePolicyTest, SchemaApiFailureTest, OverseerStatusTest, TriLevelCompositeIdRoutingTest, LargeFieldTest, TestMaxTokenLenTokenizer, TestDocumentBuilder, TestMinMaxOnMultiValuedField, ZkSolrClientTest, TestFieldResource, TestCrossCoreJoin, TestCSVResponseWriter, ConcurrentDeleteAndCreateCollectionTest, DistributedFacetPivotLargeTest, TestFieldCache, HdfsWriteToMultipleCollectionsTest, TestGenericDistributedQueue, TestSort, TestRemoteStreaming, CurrencyFieldTypeTest, TestClassicSimilarityFactory, TestConfig, DistribDocExpirationUpdateProcessorTest, HdfsDirectoryTest, CdcrRequestHandlerTest, TestRawTransformer, InfixSuggestersTest, TestJsonRequest, CdcrBootstrapTest, TestDynamicFieldCollectionResource, TestPullReplicaErrorHandling, TestSubQueryTransformerDistrib, TestJoin, TestCloudPseudoReturnFields, ZkNodePropsTest, TestBinaryField, TestPolicyCloud, TestComplexPhraseLeadingWildcard, TestLegacyFieldCache, FullSolrCloudDistribCmdsTest, SharedFSAutoReplicaFailoverTest, NodeMutatorTest, SolrRequestParserTest, DistributedTermsComponentTest, SpellCheckCollatorTest, TestComponentsName, AnalyticsQueryTest, TestNoOpRegenerator, TestOmitPositions, ZkCLITest, TestSchemaManager, TestRandomFaceting, TestCollectionsAPIViaSolrCloudCluster, LIRRollingUpdatesTest, OutOfBoxZkACLAndCredentialsProvidersTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerCollectionConfigSetProcessorTest, OverseerModifyCollectionTest, OverseerRolesTest, PeerSyncReplicationTest, ReplaceNodeTest, TestCloudInspectUtil, TestCloudRecovery, TestLeaderElectionWithEmptyReplica, TestRebalanceLeaders, TestShortCircuitedRequests, TestTolerantUpdateProcessorRandomCloud, AssignTest, CollectionsAPIDistributedZkTest, AutoScalingHandlerTest, HttpTriggerListenerTest, ScheduledTriggerTest, SystemLogListenerTest, TriggerIntegrationTest]
[junit4] Completed [653/779 (1!)] on J0 in 294.91s, 15 tests, 1 failure <<< FAILURES!
[...truncated 25896 lines...]
-check-forbidden-all:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.8
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.8
[forbidden-apis] Reading bundled API signatures: jdk-non-portable
[forbidden-apis] Reading bundled API signatures: jdk-reflection
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.5
[forbidden-apis] Reading API signatures: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Reading API signatures: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/forbiddenApis/solr.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning classes for violations...
[forbidden-apis] Forbidden method invocation: java.lang.System#currentTimeMillis() [Use RTimer/TimeOut/System.nanoTime for time comparisons, and `new Date()` output/debugging/stats of timestamps. If for some miscellaneous reason, you absolutely need to use this, use a SuppressForbidden.]
[forbidden-apis] in org.apache.solr.cloud.autoscaling.ScheduledTriggerTest (ScheduledTriggerTest.java:80)
[forbidden-apis] Scanned 3978 class file(s) for forbidden API invocations (in 3.42s), 1 error(s).
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:618: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:117: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:376: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/common-build.xml:535: Check for forbidden API calls failed, see log.
Total time: 83 minutes 7 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-master - Build # 2357 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2357/
2 tests failed.
FAILED: org.apache.solr.cloud.AddReplicaTest.test
Error Message:
core_node6:{"core":"addreplicatest_coll_shard1_replica_n5","base_url":"http://127.0.0.1:53886/solr","node_name":"127.0.0.1:53886_solr","state":"active","type":"NRT"}
Stack Trace:
java.lang.AssertionError: core_node6:{"core":"addreplicatest_coll_shard1_replica_n5","base_url":"http://127.0.0.1:53886/solr","node_name":"127.0.0.1:53886_solr","state":"active","type":"NRT"}
at __randomizedtesting.SeedInfo.seed([389B7E4FA735EC54:B0CF419509C981AC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.AddReplicaTest.test(AddReplicaTest.java:84)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger
Error Message:
expected:<2> but was:<1>
Stack Trace:
java.lang.AssertionError: expected:<2> but was:<1>
at __randomizedtesting.SeedInfo.seed([389B7E4FA735EC54:AB80363DF9C8B760]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.autoscaling.TriggerIntegrationTest.testScheduledTrigger(TriggerIntegrationTest.java:1679)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12895 lines...]
[junit4] Suite: org.apache.solr.cloud.AddReplicaTest
[junit4] 2> 840661 INFO (SUITE-AddReplicaTest-seed#[389B7E4FA735EC54]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.AddReplicaTest_389B7E4FA735EC54-001/init-core-data-001
[junit4] 2> 840662 WARN (SUITE-AddReplicaTest-seed#[389B7E4FA735EC54]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=25 numCloses=25
[junit4] 2> 840662 INFO (SUITE-AddReplicaTest-seed#[389B7E4FA735EC54]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 840669 INFO (SUITE-AddReplicaTest-seed#[389B7E4FA735EC54]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 840670 INFO (SUITE-AddReplicaTest-seed#[389B7E4FA735EC54]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.AddReplicaTest_389B7E4FA735EC54-001/tempDir-001
[junit4] 2> 840670 INFO (SUITE-AddReplicaTest-seed#[389B7E4FA735EC54]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 840673 INFO (Thread-2236) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 840673 INFO (Thread-2236) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 840680 ERROR (Thread-2236) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 840773 INFO (SUITE-AddReplicaTest-seed#[389B7E4FA735EC54]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:50291
[junit4] 2> 840826 INFO (zkConnectionManagerCallback-2743-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 840868 INFO (jetty-launcher-2740-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 840881 INFO (jetty-launcher-2740-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 841024 INFO (jetty-launcher-2740-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 841024 INFO (jetty-launcher-2740-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T14:27:37-07:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 841025 INFO (jetty-launcher-2740-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 841025 INFO (jetty-launcher-2740-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 841025 INFO (jetty-launcher-2740-thread-4) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 841025 INFO (jetty-launcher-2740-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@40b3609b{/solr,null,AVAILABLE}
[junit4] 2> 841026 INFO (jetty-launcher-2740-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@35a28d{HTTP/1.1,[http/1.1]}{127.0.0.1:59231}
[junit4] 2> 841026 INFO (jetty-launcher-2740-thread-4) [ ] o.e.j.s.Server Started @842995ms
[junit4] 2> 841026 INFO (jetty-launcher-2740-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59231}
[junit4] 2> 841026 ERROR (jetty-launcher-2740-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 841026 INFO (jetty-launcher-2740-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 841027 INFO (jetty-launcher-2740-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 841027 INFO (jetty-launcher-2740-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 841027 INFO (jetty-launcher-2740-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T08:10:44.683Z
[junit4] 2> 841087 INFO (jetty-launcher-2740-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 841087 INFO (jetty-launcher-2740-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 841087 INFO (jetty-launcher-2740-thread-3) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 841089 INFO (jetty-launcher-2740-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7f326665{/solr,null,AVAILABLE}
[junit4] 2> 841090 INFO (jetty-launcher-2740-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@66e47b8a{HTTP/1.1,[http/1.1]}{127.0.0.1:34802}
[junit4] 2> 841090 INFO (jetty-launcher-2740-thread-3) [ ] o.e.j.s.Server Started @843059ms
[junit4] 2> 841090 INFO (jetty-launcher-2740-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34802}
[junit4] 2> 841090 ERROR (jetty-launcher-2740-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 841090 INFO (jetty-launcher-2740-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 841090 INFO (jetty-launcher-2740-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 841090 INFO (jetty-launcher-2740-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 841090 INFO (jetty-launcher-2740-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T08:10:44.746Z
[junit4] 2> 841104 INFO (zkConnectionManagerCallback-2747-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841104 INFO (jetty-launcher-2740-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 841125 INFO (jetty-launcher-2740-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 841125 INFO (jetty-launcher-2740-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 841125 INFO (jetty-launcher-2740-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 841127 INFO (jetty-launcher-2740-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 841127 INFO (jetty-launcher-2740-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 841127 INFO (jetty-launcher-2740-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 841127 INFO (jetty-launcher-2740-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4ce6a5ec{/solr,null,AVAILABLE}
[junit4] 2> 841127 INFO (jetty-launcher-2740-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1fa6ab55{HTTP/1.1,[http/1.1]}{127.0.0.1:53886}
[junit4] 2> 841127 INFO (jetty-launcher-2740-thread-1) [ ] o.e.j.s.Server Started @843096ms
[junit4] 2> 841127 INFO (jetty-launcher-2740-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=53886}
[junit4] 2> 841128 ERROR (jetty-launcher-2740-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 841128 INFO (jetty-launcher-2740-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 841128 INFO (jetty-launcher-2740-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 841128 INFO (jetty-launcher-2740-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 841128 INFO (jetty-launcher-2740-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T08:10:44.784Z
[junit4] 2> 841135 INFO (zkConnectionManagerCallback-2745-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841135 INFO (jetty-launcher-2740-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 841151 INFO (jetty-launcher-2740-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50291/solr
[junit4] 2> 841151 INFO (jetty-launcher-2740-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b5cef90{/solr,null,AVAILABLE}
[junit4] 2> 841152 INFO (jetty-launcher-2740-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@353f4067{HTTP/1.1,[http/1.1]}{127.0.0.1:54455}
[junit4] 2> 841152 INFO (jetty-launcher-2740-thread-2) [ ] o.e.j.s.Server Started @843121ms
[junit4] 2> 841152 INFO (jetty-launcher-2740-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=54455}
[junit4] 2> 841152 ERROR (jetty-launcher-2740-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 841152 INFO (jetty-launcher-2740-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 841152 INFO (jetty-launcher-2740-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 841152 INFO (jetty-launcher-2740-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 841152 INFO (jetty-launcher-2740-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-22T08:10:44.808Z
[junit4] 2> 841155 INFO (zkConnectionManagerCallback-2749-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841155 INFO (jetty-launcher-2740-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 841206 INFO (zkConnectionManagerCallback-2755-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841207 INFO (jetty-launcher-2740-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 841207 INFO (jetty-launcher-2740-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50291/solr
[junit4] 2> 841214 INFO (zkConnectionManagerCallback-2753-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841236 INFO (jetty-launcher-2740-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50291/solr
[junit4] 2> 841274 INFO (zkConnectionManagerCallback-2759-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841275 DEBUG (jetty-launcher-2740-thread-3) [n:127.0.0.1:34802_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$273/1984351120@7c8b1f20
[junit4] 2> 841275 DEBUG (jetty-launcher-2740-thread-4) [n:127.0.0.1:59231_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$273/1984351120@68babcd2
[junit4] 2> 841298 INFO (jetty-launcher-2740-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:50291/solr
[junit4] 2> 841304 INFO (zkConnectionManagerCallback-2763-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841318 INFO (zkConnectionManagerCallback-2771-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841320 DEBUG (jetty-launcher-2740-thread-1) [n:127.0.0.1:53886_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$273/1984351120@188b9094
[junit4] 2> 841340 DEBUG (jetty-launcher-2740-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.c.ZkController Added new OnReconnect listener org.apache.solr.cloud.ZkController$$Lambda$273/1984351120@5a7f3e78
[junit4] 2> 841368 INFO (zkConnectionManagerCallback-2767-thread-1-processing-n:127.0.0.1:59231_solr) [n:127.0.0.1:59231_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841374 INFO (zkConnectionManagerCallback-2765-thread-1-processing-n:127.0.0.1:34802_solr) [n:127.0.0.1:34802_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841444 INFO (zkConnectionManagerCallback-2773-thread-1-processing-n:127.0.0.1:53886_solr) [n:127.0.0.1:53886_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841462 INFO (zkConnectionManagerCallback-2775-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 841498 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
[junit4] 2> 841498 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841499 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 841500 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 841500 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 841501 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 841526 INFO (jetty-launcher-2740-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 841527 DEBUG (jetty-launcher-2740-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73338098767036428-127.0.0.1:54455_solr-n_0000000000
[junit4] 2> 841528 INFO (jetty-launcher-2740-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:54455_solr
[junit4] 2> 841530 INFO (jetty-launcher-2740-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer Overseer (id=73338098767036428-127.0.0.1:54455_solr-n_0000000000) starting
[junit4] 2> 841546 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
[junit4] 2> 841546 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841547 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 841547 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 841547 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 841547 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 841548 INFO (jetty-launcher-2740-thread-1) [n:127.0.0.1:53886_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 841548 DEBUG (jetty-launcher-2740-thread-1) [n:127.0.0.1:53886_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73338098767036427-127.0.0.1:53886_solr-n_0000000001
[junit4] 2> 841549 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/73338098767036428-127.0.0.1:54455_solr-n_0000000000
[junit4] 2> 841549 DEBUG (jetty-launcher-2740-thread-1) [n:127.0.0.1:53886_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/73338098767036428-127.0.0.1:54455_solr-n_0000000000 to know if I could be the leader
[junit4] 2> 841549 INFO (jetty-launcher-2740-thread-1) [n:127.0.0.1:53886_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:53886_solr
[junit4] 2> 841549 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 841549 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 841574 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841575 INFO (zkCallback-2772-thread-1-processing-n:127.0.0.1:53886_solr) [n:127.0.0.1:53886_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 841592 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor Process current queue of overseer operations
[junit4] 2> 841598 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer Starting to work on the main queue
[junit4] 2> 841604 INFO (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 841606 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue-work
[junit4] 2> 841607 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 841607 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 0
[junit4] 2> 841607 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor RunningTasks: []
[junit4] 2> 841607 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 841608 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 841608 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: []
[junit4] 2> 841608 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskQueue Peeking for top 100 elements. ExcludeSet: []
[junit4] 2> 841609 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 841619 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
[junit4] 2> 841619 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841620 INFO (jetty-launcher-2740-thread-3) [n:127.0.0.1:34802_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 841620 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 841620 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 841620 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 841621 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 841622 INFO (jetty-launcher-2740-thread-3) [n:127.0.0.1:34802_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 841622 DEBUG (jetty-launcher-2740-thread-3) [n:127.0.0.1:34802_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73338098767036425-127.0.0.1:34802_solr-n_0000000002
[junit4] 2> 841622 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/73338098767036427-127.0.0.1:53886_solr-n_0000000001
[junit4] 2> 841622 DEBUG (jetty-launcher-2740-thread-3) [n:127.0.0.1:34802_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/73338098767036427-127.0.0.1:53886_solr-n_0000000001 to know if I could be the leader
[junit4] 2> 841625 DEBUG (jetty-launcher-2740-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:54455_solr as DOWN
[junit4] 2> 841625 DEBUG (jetty-launcher-2740-thread-3) [n:127.0.0.1:34802_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:34802_solr as DOWN
[junit4] 2> 841626 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 841626 INFO (jetty-launcher-2740-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:54455_solr
[junit4] 2> 841626 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 841626 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 841631 DEBUG (OverseerAutoScalingTriggerThread-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 841632 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841632 INFO (zkCallback-2774-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 841632 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/autoscaling.json
[junit4] 2> 841632 DEBUG (OverseerAutoScalingTriggerThread-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 841632 DEBUG (OverseerAutoScalingTriggerThread-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:54455_solr, 127.0.0.1:53886_solr]
[junit4] 2> 841632 DEBUG (OverseerAutoScalingTriggerThread-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 841632 DEBUG (OverseerAutoScalingTriggerThread-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 841637 INFO (jetty-launcher-2740-thread-3) [n:127.0.0.1:34802_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34802_solr
[junit4] 2> 841638 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841638 INFO (zkCallback-2772-thread-1-processing-n:127.0.0.1:53886_solr) [n:127.0.0.1:53886_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 841638 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 841638 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 841639 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841639 INFO (zkCallback-2772-thread-1-processing-n:127.0.0.1:53886_solr) [n:127.0.0.1:53886_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 841685 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
[junit4] 2> 841686 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841686 INFO (jetty-launcher-2740-thread-4) [n:127.0.0.1:59231_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 841686 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 841686 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 841686 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 841686 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/security.json
[junit4] 2> 841687 INFO (jetty-launcher-2740-thread-4) [n:127.0.0.1:59231_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 841692 DEBUG (jetty-launcher-2740-thread-4) [n:127.0.0.1:59231_solr ] o.a.s.c.LeaderElector Joined leadership election with path: /overseer_elect/election/73338098767036426-127.0.0.1:59231_solr-n_0000000003
[junit4] 2> 841692 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/overseer_elect/election/73338098767036425-127.0.0.1:34802_solr-n_0000000002
[junit4] 2> 841693 DEBUG (jetty-launcher-2740-thread-4) [n:127.0.0.1:59231_solr ] o.a.s.c.LeaderElector Watching path /overseer_elect/election/73338098767036425-127.0.0.1:34802_solr-n_0000000002 to know if I could be the leader
[junit4] 2> 841693 DEBUG (jetty-launcher-2740-thread-4) [n:127.0.0.1:59231_solr ] o.a.s.c.ZkController Publish node=127.0.0.1:59231_solr as DOWN
[junit4] 2> 841700 INFO (jetty-launcher-2740-thread-4) [n:127.0.0.1:59231_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59231_solr
[junit4] 2> 841701 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 841701 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/live_nodes
[junit4] 2> 841701 DEBUG (OverseerAutoScalingTriggerThread-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 841701 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841701 INFO (zkCallback-2772-thread-1-processing-n:127.0.0.1:53886_solr) [n:127.0.0.1:53886_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 841702 DEBUG (OverseerAutoScalingTriggerThread-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 841706 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 841706 DEBUG (ScheduledTrigger-4551-thread-1) [n:127.0.0.1:54455_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 841720 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841720 INFO (zkCallback-2764-thread-1-processing-n:127.0.0.1:34802_solr) [n:127.0.0.1:34802_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (4)
[junit4] 2> 841777 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841777 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 841778 INFO (zkCallback-2774-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (4)
[junit4] 2> 841778 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 841779 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:54455_solr"} current state version: 0
[junit4] 2> 841779 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:54455_solr
[junit4] 2> 841779 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:34802_solr"} current state version: 0
[junit4] 2> 841779 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:34802_solr
[junit4] 2> 841779 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer processMessage: queueSize: 3, message = {
[junit4] 2> "operation":"downnode",
[junit4] 2> "node_name":"127.0.0.1:59231_solr"} current state version: 0
[junit4] 2> 841779 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.NodeMutator DownNode state invoked for node: 127.0.0.1:59231_solr
[junit4] 2> 841787 INFO (jetty-launcher-2740-thread-1) [n:127.0.0.1:53886_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53886.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 841788 INFO (zkCallback-2766-thread-1-processing-n:127.0.0.1:59231_solr) [n:127.0.0.1:59231_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 841824 INFO (jetty-launcher-2740-thread-1) [n:127.0.0.1:53886_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53886.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 841824 INFO (jetty-launcher-2740-thread-1) [n:127.0.0.1:53886_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_53886.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 841825 INFO (jetty-launcher-2740-thread-1) [n:127.0.0.1:53886_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.AddReplicaTest_389B7E4FA735EC54-001/tempDir-001/node1/.
[junit4] 2> 841879 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 841964 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 842001 INFO (jetty-launcher-2740-thread-3) [n:127.0.0.1:34802_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34802.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 842022 INFO (jetty-launcher-2740-thread-3) [n:127.0.0.1:34802_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34802.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 842022 INFO (jetty-launcher-2740-thread-3) [n:127.0.0.1:34802_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34802.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 842023 INFO (jetty-launcher-2740-thread-3) [n:127.0.0.1:34802_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.AddReplicaTest_389B7E4FA735EC54-001/tempDir-001/node3/.
[junit4] 2> 842028 INFO (jetty-launcher-2740-thread-4) [n:127.0.0.1:59231_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59231.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 842038 INFO (jetty-launcher-2740-thread-4) [n:127.0.0.1:59231_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59231.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 842038 INFO (jetty-launcher-2740-thread-4) [n:127.0.0.1:59231_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59231.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 842039 INFO (jetty-launcher-2740-thread-4) [n:127.0.0.1:59231_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.AddReplicaTest_389B7E4FA735EC54-001/tempDir-001/node4/.
[junit4] 2> 842076 INFO (jetty-launcher-2740-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54455.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 842097 INFO (jetty-launcher-2740-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54455.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 842097 INFO (jetty-launcher-2740-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54455.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 842098 INFO (jetty-launcher-2740-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.AddReplicaTest_389B7E4FA735EC54-001/tempDir-001/node2/.
[junit4] 2> 842238 INFO (zkConnectionManagerCallback-2785-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 842240 INFO (zkConnectionManagerCallback-2789-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 842241 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/clusterprops.json
[junit4] 2> 842241 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/live_nodes
[junit4] 2> 842241 INFO (SUITE-AddReplicaTest-seed#[389B7E4FA735EC54]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 842241 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/clusterstate.json
[junit4] 2> 842242 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 842242 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/aliases.json
[junit4] 2> 842242 INFO (SUITE-AddReplicaTest-seed#[389B7E4FA735EC54]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:50291/solr ready
[junit4] 2> 842282 INFO (TEST-AddReplicaTest.test-seed#[389B7E4FA735EC54]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 842290 INFO (zkConnectionManagerCallback-2791-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 842302 INFO (qtp931830872-11409) [n:127.0.0.1:34802_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&maxShardsPerNode=2&name=addreplicatest_coll&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 842302 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
[junit4] 2> 842302 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 842302 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
[junit4] 2> 842303 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/overseer/collection-queue-work/qnr-0000000000
[junit4] 2> 842303 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/collection-queue-work
[junit4] 2> 842303 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/collection-queue-work
[junit4] 2> 842303 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.ZkDistributedQueue Found child node with improper name: qnr-0000000000
[junit4] 2> 842304 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskQueue Returning topN elements: [/overseer/collection-queue-work/qn-0000000000, ]
[junit4] 2> 842304 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor Got 1 tasks from work-queue : [[org.apache.solr.cloud.OverseerTaskQueue$QueueEvent@2e684ecf]]
[junit4] 2> 842304 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor Marked task [/overseer/collection-queue-work/qn-0000000000] as running
[junit4] 2> 842304 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor Overseer Collection Message Handler: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
[junit4] 2> "name":"addreplicatest_coll",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"}
[junit4] 2> 842322 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor Cleaning up work-queue. #Running tasks: 1
[junit4] 2> 842322 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor RunningTasks: [/overseer/collection-queue-work/qn-0000000000]
[junit4] 2> 842322 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor BlockedTasks: []
[junit4] 2> 842322 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor CompletedTasks: []
[junit4] 2> 842322 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor RunningZKTasks: [/overseer/collection-queue-work/qn-0000000000]
[junit4] 2> 842322 DEBUG (OverseerCollectionConfigSetProcessor-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskQueue Peeking for top 99 elements. ExcludeSet: [/overseer/collection-queue-work/qn-0000000000],[]
[junit4] 2> 842322 DEBUG (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.OverseerTaskProcessor Runner processing /overseer/collection-queue-work/qn-0000000000
[junit4] 2> 842322 DEBUG (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler OverseerCollectionMessageHandler.processMessage : create , {
[junit4] 2> "name":"addreplicatest_coll",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"}
[junit4] 2> 842322 INFO (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection addreplicatest_coll
[junit4] 2> 842323 DEBUG (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.a.c.Assign Identify nodes using default
[junit4] 2> 842323 DEBUG (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler creating collections conf node /collections/addreplicatest_coll
[junit4] 2> 842324 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 842324 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 842324 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 842325 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 842325 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/collections
[junit4] 2> 842325 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 842325 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 842325 DEBUG (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.a.c.CreateCollectionCmd Check for collection zkNode:addreplicatest_coll
[junit4] 2> 842326 DEBUG (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.a.c.CreateCollectionCmd Collection zkNode exists
[junit4] 2> 842326 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 842327 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 842327 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "name":"addreplicatest_coll",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"} current state version: 0
[junit4] 2> 842327 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.ClusterStateMutator building a new cName: addreplicatest_coll
[junit4] 2> 842327 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.ZkStateWriter going to create_collection /collections/addreplicatest_coll/state.json
[junit4] 2> 842328 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 842328 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 842328 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 842329 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 842329 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/collections
[junit4] 2> 842555 DEBUG (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.a.c.CreateCollectionCmd Creating SolrCores for new collection addreplicatest_coll, shardNames [shard1, shard2] , message : {
[junit4] 2> "name":"addreplicatest_coll",
[junit4] 2> "fromApi":"true",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "collection.configName":"conf1",
[junit4] 2> "numShards":"2",
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "stateFormat":"2",
[junit4] 2> "operation":"create"}
[junit4] 2> 842558 DEBUG (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.a.c.CreateCollectionCmd Creating core addreplicatest_coll_shard1_replica_n1 as part of shard shard1 of collection addreplicatest_coll on 127.0.0.1:34802_solr
[junit4] 2> 842562 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 842563 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 842563 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34802/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 842563 INFO (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:34802/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 842565 DEBUG (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.a.c.CreateCollectionCmd Creating core addreplicatest_coll_shard2_replica_n2 as part of shard shard2 of collection addreplicatest_coll on 127.0.0.1:54455_solr
[junit4] 2> 842566 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.SliceMutator Old Slice: shard1:{
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 842566 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.SliceMutator New Slice: shard1:{
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:34802/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 842566 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 842566 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 842567 DEBUG (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Expecting [addreplicatest_coll_shard1_replica_n1, addreplicatest_coll_shard2_replica_n2] cores but found {}
[junit4] 2> 842567 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:54455/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"} current state version: 0
[junit4] 2> 842567 INFO (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:54455/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 842567 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.SliceMutator Old Slice: shard2:{
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{}}
[junit4] 2> 842568 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.SliceMutator New Slice: shard2:{
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node4":{
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:54455/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}}
[junit4] 2> 842667 DEBUG (OverseerThreadFactory-4553-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.a.c.OverseerCollectionMessageHandler Expecting [addreplicatest_coll_shard1_replica_n1, addreplicatest_coll_shard2_replica_n2] cores but found {}
[junit4] 2> 842668 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/addreplicatest_coll/state.json version: 0
[junit4] 2> 842669 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 842669 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 842708 DEBUG (ScheduledTrigger-4551-thread-1) [n:127.0.0.1:54455_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 842776 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&newCollection=true&name=addreplicatest_coll_shard2_replica_n2&action=CREATE&numShards=2&collection=addreplicatest_coll&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 842776 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 842777 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 842778 DEBUG (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.c.ZkController publishing state=down
[junit4] 2> 842778 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 842778 DEBUG (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.c.ZkController Registering watch for collection addreplicatest_coll
[junit4] 2> 842778 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 842779 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=addreplicatest_coll_shard1_replica_n1&action=CREATE&numShards=2&collection=addreplicatest_coll&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 842779 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 842779 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 842779 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n2",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:54455/solr",
[junit4] 2> "node_name":"127.0.0.1:54455_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 842780 DEBUG (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.c.ZkController waiting to find shard id in clusterstate for addreplicatest_coll_shard2_replica_n2
[junit4] 2> 842780 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n2",
[junit4] 2> "core_node_name":"core_node4",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:54455/solr",
[junit4] 2> "node_name":"127.0.0.1:54455_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard2",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 842780 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node4:{"core":"addreplicatest_coll_shard2_replica_n2","base_url":"http://127.0.0.1:54455/solr","node_name":"127.0.0.1:54455_solr","state":"down","type":"NRT"}
[junit4] 2> 842780 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(addreplicatest_coll//collections/addreplicatest_coll/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{
[junit4] 2> "shard1":{
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:34802/solr",
[junit4] 2> "state":"down",
[junit4] 2> "node_name":null,
[junit4] 2> "type":"NRT"}}},
[junit4] 2> "shard2":{
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node4":{
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:54455/solr",
[junit4] 2> "node_name":"127.0.0.1:54455_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 842781 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 842785 DEBUG (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ZkController publishing state=down
[junit4] 2> 842785 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 842785 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 842786 DEBUG (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ZkController Registering watch for collection addreplicatest_coll
[junit4] 2> 842786 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 842786 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer processMessage: queueSize: 2, message = {
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:34802/solr",
[junit4] 2> "node_name":"127.0.0.1:34802_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"} current state version: 0
[junit4] 2> 842786 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.ReplicaMutator Update state numShards=2 message={
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "core_node_name":"core_node3",
[junit4] 2> "roles":null,
[junit4] 2> "base_url":"http://127.0.0.1:34802/solr",
[junit4] 2> "node_name":"127.0.0.1:34802_solr",
[junit4] 2> "numShards":"2",
[junit4] 2> "state":"down",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "type":"NRT",
[junit4] 2> "operation":"state"}
[junit4] 2> 842786 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.ReplicaMutator Will update state for replica: core_node3:{"core":"addreplicatest_coll_shard1_replica_n1","base_url":"http://127.0.0.1:34802/solr","node_name":"127.0.0.1:34802_solr","state":"down","type":"NRT"}
[junit4] 2> 842786 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.ReplicaMutator Collection is now: DocCollection(addreplicatest_coll//collections/addreplicatest_coll/state.json/1)={
[junit4] 2> "pullReplicas":"0",
[junit4] 2> "replicationFactor":"1",
[junit4] 2> "router":{"name":"compositeId"},
[junit4] 2> "maxShardsPerNode":"2",
[junit4] 2> "autoAddReplicas":"false",
[junit4] 2> "nrtReplicas":"1",
[junit4] 2> "tlogReplicas":"0",
[junit4] 2> "shards":{
[junit4] 2> "shard1":{
[junit4] 2> "range":"80000000-ffffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node3":{
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "base_url":"http://127.0.0.1:34802/solr",
[junit4] 2> "node_name":"127.0.0.1:34802_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}},
[junit4] 2> "shard2":{
[junit4] 2> "range":"0-7fffffff",
[junit4] 2> "state":"active",
[junit4] 2> "replicas":{"core_node4":{
[junit4] 2> "core":"addreplicatest_coll_shard2_replica_n2",
[junit4] 2> "base_url":"http://127.0.0.1:54455/solr",
[junit4] 2> "node_name":"127.0.0.1:54455_solr",
[junit4] 2> "state":"down",
[junit4] 2> "type":"NRT"}}}}}
[junit4] 2> 842792 DEBUG (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ZkController waiting to find shard id in clusterstate for addreplicatest_coll_shard1_replica_n1
[junit4] 2> 842887 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.o.ZkStateWriter going to update_collection /collections/addreplicatest_coll/state.json version: 1
[junit4] 2> 842887 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 842887 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 842887 INFO (zkCallback-2764-thread-1-processing-n:127.0.0.1:34802_solr) [n:127.0.0.1:34802_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addreplicatest_coll/state.json] for collection [addreplicatest_coll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 842887 INFO (zkCallback-2774-thread-1-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addreplicatest_coll/state.json] for collection [addreplicatest_coll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 842888 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 842888 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 842888 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 842900 INFO (zkCallback-2774-thread-2-processing-n:127.0.0.1:54455_solr) [n:127.0.0.1:54455_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addreplicatest_coll/state.json] for collection [addreplicatest_coll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 842900 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 842900 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 842901 INFO (zkCallback-2764-thread-2-processing-n:127.0.0.1:34802_solr) [n:127.0.0.1:34802_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/addreplicatest_coll/state.json] for collection [addreplicatest_coll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 842901 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: data: /solr/collections/addreplicatest_coll/state.json
[junit4] 2> 843712 DEBUG (ScheduledTrigger-4551-thread-2) [n:127.0.0.1:54455_solr ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 843789 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 843798 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 843809 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.s.IndexSchema [addreplicatest_coll_shard1_replica_n1] Schema name=minimal
[junit4] 2> 843810 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.s.IndexSchema [addreplicatest_coll_shard2_replica_n2] Schema name=minimal
[junit4] 2> 843814 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 843814 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'addreplicatest_coll_shard2_replica_n2' using configuration from collection addreplicatest_coll, trusted=true
[junit4] 2> 843814 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_54455.solr.core.addreplicatest_coll.shard2.replica_n2' (registry 'solr.core.addreplicatest_coll.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 843815 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 843815 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.c.SolrCore [[addreplicatest_coll_shard2_replica_n2] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.AddReplicaTest_389B7E4FA735EC54-001/tempDir-001/node2/addreplicatest_coll_shard2_replica_n2], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.AddReplicaTest_389B7E4FA735EC54-001/tempDir-001/node2/./addreplicatest_coll_shard2_replica_n2/data/]
[junit4] 2> 843824 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 843824 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'addreplicatest_coll_shard1_replica_n1' using configuration from collection addreplicatest_coll, trusted=true
[junit4] 2> 843827 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34802.solr.core.addreplicatest_coll.shard1.replica_n1' (registry 'solr.core.addreplicatest_coll.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@717e75de
[junit4] 2> 843827 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 843827 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.SolrCore [[addreplicatest_coll_shard1_replica_n1] ] Opening new SolrCore at [/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.AddReplicaTest_389B7E4FA735EC54-001/tempDir-001/node3/addreplicatest_coll_shard1_replica_n1], dataDir=[/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.AddReplicaTest_389B7E4FA735EC54-001/tempDir-001/node3/./addreplicatest_coll_shard1_replica_n1/data/]
[junit4] 2> 844153 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 844153 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 844154 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 844154 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 844156 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@a13f81b[addreplicatest_coll_shard1_replica_n1] main]
[junit4] 2> 844157 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 844158 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 844169 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 844170 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593087935461195776
[junit4] 2> 844173 INFO (searcherExecutor-4563-thread-1-processing-n:127.0.0.1:34802_solr x:addreplicatest_coll_shard1_replica_n1 s:shard1 c:addreplicatest_coll r:core_node3) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.SolrCore [addreplicatest_coll_shard1_replica_n1] Registered new searcher Searcher@a13f81b[addreplicatest_coll_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 844174 DEBUG (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ZkController watch zkdir /configs/conf1
[junit4] 2> 844174 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/configs/conf1
[junit4] 2> 844178 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/collections/addreplicatest_coll/terms/shard1
[junit4] 2> 844178 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: data: /solr/collections/addreplicatest_coll/terms/shard1
[junit4] 2> 844178 DEBUG (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ZkController Register replica - core:addreplicatest_coll_shard1_replica_n1 address:http://127.0.0.1:34802/solr collection:addreplicatest_coll shard:shard1
[junit4] 2> 844180 DEBUG (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.LeaderElector Joined leadership election with path: /collections/addreplicatest_coll/leader_elect/shard1/election/73338098767036425-core_node3-n_0000000000
[junit4] 2> 844181 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/collections/addreplicatest_coll/terms/shard1
[junit4] 2> 844181 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 844181 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 844181 DEBUG (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Running the leader process for shard=shard1 and weAreReplacement=false and leaderVoteWait=10000
[junit4] 2> 844182 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 844182 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 844182 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34802/solr/addreplicatest_coll_shard1_replica_n1/
[junit4] 2> 844182 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 844182 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:34802/solr/addreplicatest_coll_shard1_replica_n1/ has no replicas
[junit4] 2> 844182 DEBUG (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext 127.0.0.1:34802_solr synched 0
[junit4] 2> 844182 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 844183 DEBUG (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContextBase Creating leader registration node /collections/addreplicatest_coll/leaders/shard1/leader after winning as /collections/addreplicatest_coll/leader_elect/shard1/election/73338098767036425-core_node3-n_0000000000
[junit4] 2> 844185 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: children: /solr/overseer/queue
[junit4] 2> 844186 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: children: /solr/overseer/queue
[junit4] 2> 844186 DEBUG (OverseerStateUpdate-73338098767036428-127.0.0.1:54455_solr-n_0000000000) [n:127.0.0.1:54455_solr ] o.a.s.c.Overseer processMessage: queueSize: 1, message = {
[junit4] 2> "operation":"leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"addreplicatest_coll",
[junit4] 2> "base_url":"http://127.0.0.1:34802/solr",
[junit4] 2> "core":"addreplicatest_coll_shard1_replica_n1",
[junit4] 2> "state":"active"} current state version: 0
[junit4] 2> 844186 INFO (qtp931830872-11413) [n:127.0.0.1:34802_solr c:addreplicatest_coll s:shard1 r:core_node3 x:addreplicatest_coll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34802/solr/addreplicatest_coll_shard1_replica_n1/ shard1
[junit4] 2> 844198 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 844198 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 844202 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@43469c90[addreplicatest_coll_shard2_replica_n2] main]
[junit4] 2> 844207 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 844207 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 844208 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 844209 INFO (searcherExecutor-4562-thread-1-processing-n:127.0.0.1:54455_solr x:addreplicatest_coll_shard2_replica_n2 s:shard2 c:addreplicatest_coll r:core_node4) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.c.SolrCore [addreplicatest_coll_shard2_replica_n2] Registered new searcher Searcher@43469c90[addreplicatest_coll_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 844210 INFO (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1593087935503138816
[junit4] 2> 844212 DEBUG (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.c.ZkController watch zkdir /configs/conf1
[junit4] 2> 844212 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/configs/conf1
[junit4] 2> 844220 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/collections/addreplicatest_coll/terms/shard2
[junit4] 2> 844222 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch fired: data: /solr/collections/addreplicatest_coll/terms/shard2
[junit4] 2> 844222 DEBUG (SyncThread:0) [ ] o.a.s.c.ZkTestServer Watch added: create/delete: /solr/collections/addreplicatest_coll/terms/shard2
[junit4] 2> 844222 DEBUG (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.c.ZkController Register replica - core:addreplicatest_coll_shard2_replica_n2 address:http://127.0.0.1:54455/solr collection:addreplicatest_coll shard:shard2
[junit4] 2> 844226 DEBUG (qtp1457128976-11421) [n:127.0.0.1:54455_solr c:addreplicatest_coll s:shard2 r:core_node4 x:addreplicatest_coll_shard2_replica_n2] o.a.s.c.LeaderElector Joined leadership election with path: /collections/addreplicatest_coll/leader_el
[...truncated too long message...]
r is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2645717 INFO (SUITE-TriggerIntegrationTest-seed#[389B7E4FA735EC54]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40832 40832
[junit4] 2> 2645822 INFO (Thread-4530) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40832 40832
[junit4] 2> 2645824 WARN (Thread-4530) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 43 /solr/collections/collection1/terms/shard1
[junit4] 2> 42 /solr/aliases.json
[junit4] 2> 42 /solr/clusterprops.json
[junit4] 2> 26 /solr/security.json
[junit4] 2> 22 /solr/collections/testScheduledTrigger/terms/shard2
[junit4] 2> 14 /solr/collections/testScheduledTrigger/terms/shard1
[junit4] 2> 12 /solr/collections/testMetricTrigger/terms/shard1
[junit4] 2> 9 /solr/configs/conf
[junit4] 2> 7 /solr/configs/testScheduledTrigger.AUTOCREATED/managed-schema
[junit4] 2> 6 /solr/configs/testScheduledTrigger.AUTOCREATED
[junit4] 2> 2 /solr/collections/testMetricTrigger/terms/shard2
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 65 /solr/collections/testScheduledTrigger/state.json
[junit4] 2> 64 /solr/collections/collection1/state.json
[junit4] 2> 42 /solr/clusterstate.json
[junit4] 2> 37 /solr/collections/testMetricTrigger/state.json
[junit4] 2> 21 /solr/autoscaling.json
[junit4] 2> 3 /solr/overseer_elect/election/73338197755494439-127.0.0.1:59993_solr-n_0000000011
[junit4] 2> 3 /solr/collections/collection1/leader_elect/shard1/election/73338197755494414-core_node3-n_0000000001
[junit4] 2> 2 /solr/overseer_elect/election/73338197755494476-127.0.0.1:45626_solr-n_0000000023
[junit4] 2> 2 /solr/collections/testScheduledTrigger/leader_elect/shard1/election/73338197755494436-core_node5-n_0000000000
[junit4] 2> 2 /solr/collections/testScheduledTrigger/leader_elect/shard2/election/73338197755494430-core_node9-n_0000000000
[junit4] 2> 2 /solr/collections/collection1/leader_elect/shard1/election/73338197755494405-core_node8-n_0000000003
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 430 /solr/live_nodes
[junit4] 2> 67 /solr/collections
[junit4] 2> 21 /solr/overseer/queue-work
[junit4] 2> 21 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 21 /solr/overseer/queue
[junit4] 2> 21 /solr/overseer/collection-queue-work
[junit4] 2> 2 /solr/autoscaling/events/node_lost_triggerMR
[junit4] 2> 2 /solr/autoscaling/events/node_added_triggerCTOOR
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.TriggerIntegrationTest_389B7E4FA735EC54-001
[junit4] 2> Feb 22, 2018 8:40:49 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@217fe838), locale=en, timezone=Etc/GMT-4
[junit4] 2> NOTE: Linux 3.13.0-88-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=4,threads=1,free=131929528,total=517472256
[junit4] 2> NOTE: All tests run in this JVM: [TestBM25SimilarityFactory, TestApiFramework, TestSearchPerf, TestDistributedGrouping, TestRequestForwarding, TestAddFieldRealTimeGet, TestExactSharedStatsCache, TestRecovery, TestCloudManagedSchema, ChaosMonkeySafeLeaderTest, OverseerTest, SuggesterFSTTest, TestStandardQParsers, TestSolrConfigHandlerCloud, SampleTest, DirectUpdateHandlerOptimizeTest, JsonLoaderTest, TestRetrieveFieldsOptimizer, TestTrie, HdfsAutoAddReplicasIntegrationTest, RegexBytesRefFilterTest, V2ApiIntegrationTest, OverseerTaskQueueTest, HdfsNNFailoverTest, ScriptEngineTest, TestHalfAndHalfDocValues, TestCryptoKeys, SSLMigrationTest, CloudMLTQParserTest, BasicZkTest, NodeAddedTriggerTest, BasicAuthIntegrationTest, TestReload, TestTestInjection, TestXmlQParser, DistributedFacetPivotLongTailTest, TimeRoutedAliasUpdateProcessorTest, TestSchemaResource, TestCloudJSONFacetJoinDomain, CoreAdminCreateDiscoverTest, SearchRateTriggerTest, TestCoreAdminApis, TestShortCircuitedRequests, BlockJoinFacetDistribTest, CollectionsAPIAsyncDistributedZkTest, TestNumericTerms32, RulesTest, WrapperMergePolicyFactoryTest, TestSystemIdResolver, DeleteStatusTest, TestQueryUtils, CustomCollectionTest, TestLRUCache, VMParamsZkACLAndCredentialsProvidersTest, BadComponentTest, TestSolrIndexConfig, DistribJoinFromCollectionTest, ExplicitHLLTest, TestExpandComponent, TestUnifiedSolrHighlighter, QueryEqualityTest, CreateRoutedAliasTest, TestQueryWrapperFilter, SimplePostToolTest, SecurityConfHandlerTest, TestShardHandlerFactory, PathHierarchyTokenizerFactoryTest, TestDocBasedVersionConstraints, SliceStateTest, TestFunctionQuery, OverseerModifyCollectionTest, TestTrackingShardHandlerFactory, QueryResultKeyTest, BaseCdcrDistributedZkTest, TestDynamicFieldResource, TestPostingsSolrHighlighter, SuggesterTest, SolrMetricReporterTest, TestSearcherReuse, VersionInfoTest, SchemaVersionSpecificBehaviorTest, TestMacros, CollectionStateFormat2Test, ConfigSetsAPITest, TestInitParams, TestXIncludeConfig, TestReversedWildcardFilterFactory, TestInitQParser, HLLUtilTest, HighlighterTest, QueryElevationComponentTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, FastVectorHighlighterTest, ReplicaListTransformerTest, DistributedSuggestComponentTest, TestConfigSetImmutable, TestSolr4Spatial2, MergeStrategyTest, SystemInfoHandlerTest, TestJettySolrRunner, TestLargeCluster, SolrInfoBeanTest, SolrCLIZkUtilsTest, SolrCoreTest, InfoHandlerTest, TestBulkSchemaAPI, SortByFunctionTest, TestJsonFacetRefinement, TestObjectReleaseTracker, AddReplicaTest, TestPushWriter, TestSubQueryTransformer, TestAuthorizationFramework, ZkStateWriterTest, TestLegacyFieldReuse, TestManagedSynonymFilterFactory, TestFieldTypeCollectionResource, DataDrivenBlockJoinTest, TestUtilizeNode, TestRestoreCore, HdfsRestartWhileUpdatingTest, TestPartialUpdateDeduplication, TestGeoJSONResponseWriter, TestZkChroot, TestTlogReplica, AssignTest, TestCursorMarkWithoutUniqueKey, HttpPartitionOnCommitTest, SyncSliceTest, OpenCloseCoreStressTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, ClusterStateUpdateTest, TestRandomFaceting, DistributedSpellCheckComponentTest, TestRangeQuery, DirectUpdateHandlerTest, SoftAutoCommitTest, TestRTGBase, CursorPagingTest, SolrTestCaseJ4Test, TestSimpleTrackingShardHandler, ActionThrottleTest, AsyncCallRequestStatusResponseTest, CreateCollectionCleanupTest, DeleteNodeTest, DeleteReplicaTest, DistribDocExpirationUpdateProcessorTest, HealthCheckHandlerTest, HttpPartitionTest, LIRRollingUpdatesTest, LeaderFailureAfterFreshStartTest, MissingSegmentRecoveryTest, MoveReplicaTest, RemoteQueryErrorTest, ReplaceNodeNoTargetTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCloudInspectUtil, TestCloudPivotFacet, TestConfigSetsAPI, TestLeaderElectionZkExpiry, TestLockTree, TestOnReconnectListenerSupport, TestPullReplica, TestRandomFlRTGCloud, TestRandomRequestDistribution, TestSizeLimitedDistributedMap, TestSolrCloudWithDelegationTokens, TestSolrCloudWithKerberosAlt, TestSolrCloudWithSecureImpersonation, TestTolerantUpdateProcessorCloud, ConcurrentDeleteAndCreateCollectionTest, ShardSplitTest, ComputePlanActionTest, TriggerIntegrationTest]
[junit4] Completed [625/779 (2!)] on J1 in 295.05s, 15 tests, 1 failure <<< FAILURES!
[...truncated 25396 lines...]
-check-forbidden-all:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.8
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.8
[forbidden-apis] Reading bundled API signatures: jdk-non-portable
[forbidden-apis] Reading bundled API signatures: jdk-reflection
[forbidden-apis] Reading bundled API signatures: commons-io-unsafe-2.5
[forbidden-apis] Reading API signatures: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/forbiddenApis/servlet-api.txt
[forbidden-apis] Reading API signatures: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/tools/forbiddenApis/solr.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning classes for violations...
[forbidden-apis] Forbidden method invocation: java.lang.System#currentTimeMillis() [Use RTimer/TimeOut/System.nanoTime for time comparisons, and `new Date()` output/debugging/stats of timestamps. If for some miscellaneous reason, you absolutely need to use this, use a SuppressForbidden.]
[forbidden-apis] in org.apache.solr.cloud.autoscaling.ScheduledTriggerTest (ScheduledTriggerTest.java:80)
[forbidden-apis] Scanned 3977 class file(s) for forbidden API invocations (in 3.31s), 1 error(s).
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:618: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:117: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:376: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/common-build.xml:535: Check for forbidden API calls failed, see log.
Total time: 82 minutes 22 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-master - Build # 2356 - Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2356/
6 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.ltr.feature.TestExternalValueFeatures
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestExternalValueFeatures: 1) Thread[id=24, name=qtp572864418-24, state=TIMED_WAITING, group=TGRP-TestExternalValueFeatures] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestExternalValueFeatures:
1) Thread[id=24, name=qtp572864418-24, state=TIMED_WAITING, group=TGRP-TestExternalValueFeatures]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([BFB4FE6E10EE8A02]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.ltr.feature.TestExternalValueFeatures
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=24, name=qtp572864418-24, state=TIMED_WAITING, group=TGRP-TestExternalValueFeatures] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=24, name=qtp572864418-24, state=TIMED_WAITING, group=TGRP-TestExternalValueFeatures]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([BFB4FE6E10EE8A02]:0)
FAILED: org.apache.solr.cloud.ReplaceNodeNoTargetTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([B8802F5058FD4B33:30D4108AF60126CB]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([B8802F5058FD4B33:E5C831D9973BED7C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.cdcr.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
Error Message:
Document mismatch on target after sync expected:<2000> but was:<1901>
Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1901>
at __randomizedtesting.SeedInfo.seed([B8802F5058FD4B33:6CC56409BFABF8C8]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.cdcr.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:292)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([B8802F5058FD4B33:E5C831D9973BED7C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12622 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[junit4] 2> 1662839 INFO (SUITE-TestTriggerIntegration-seed#[B8802F5058FD4B33]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_B8802F5058FD4B33-001/init-core-data-001
[junit4] 2> 1662839 INFO (SUITE-TestTriggerIntegration-seed#[B8802F5058FD4B33]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1662861 INFO (SUITE-TestTriggerIntegration-seed#[B8802F5058FD4B33]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 1662931 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1662932 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 1662932 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10000_solr]
[junit4] 2> 1662932 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 1662932 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 1662951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1662951 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 1662951 DEBUG (ScheduledTrigger-1981-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1662971 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
[junit4] 2> 1662971 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 1662971 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 1662971 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1662972 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1662972 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 1662999 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1662999 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1663018 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1663039 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1663039 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 1663039 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663039 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 1663039 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 1663040 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1663040 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 1663059 DEBUG (ScheduledTrigger-1984-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663080 DEBUG (ScheduledTrigger-1984-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663100 DEBUG (ScheduledTrigger-1984-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663121 DEBUG (ScheduledTrigger-1984-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663141 DEBUG (ScheduledTrigger-1984-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663155 DEBUG (simCloudManagerPool-1983-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1663155 DEBUG (simCloudManagerPool-1983-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 1663156 DEBUG (simCloudManagerPool-1983-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663170 DEBUG (simCloudManagerPool-1983-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663170 DEBUG (simCloudManagerPool-1983-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 1663171 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 1663171 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1663171 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 1663173 DEBUG (ScheduledTrigger-1984-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663173 DEBUG (ScheduledTrigger-1984-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1663190 DEBUG (ScheduledTrigger-1984-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1663217 DEBUG (ScheduledTrigger-1984-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663218 DEBUG (ScheduledTrigger-1984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1663218 DEBUG (ScheduledTrigger-1984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1663218 DEBUG (ScheduledTrigger-1984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10002_solr at time 1601841420580985
[junit4] 2> 1663238 DEBUG (ScheduledTrigger-1984-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1663239 DEBUG (ScheduledTrigger-1984-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1663239 DEBUG (ScheduledTrigger-1984-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1663258 DEBUG (ScheduledTrigger-1984-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1663259 DEBUG (ScheduledTrigger-1984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1663259 DEBUG (ScheduledTrigger-1984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1663278 DEBUG (ScheduledTrigger-1984-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1663279 DEBUG (ScheduledTrigger-1984-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1663279 DEBUG (ScheduledTrigger-1984-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1663279 DEBUG (ScheduledTrigger-1984-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10002_solr] added at times [1601841420580985], now=1601844471124835
[junit4] 2> 1663280 DEBUG (ScheduledTrigger-1984-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"5b0ddc4dc3879T4z9bvk1oicw7qrdzyny3ix5n8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":1601841420580985,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1601841420580985],
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 1663280 DEBUG (ScheduledTrigger-1984-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 1663281 DEBUG (AutoscalingActionExecutor-1985-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"5b0ddc4dc3879T4z9bvk1oicw7qrdzyny3ix5n8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":1601841420580985,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1601841420580985],
[junit4] 2> "_enqueue_time_":1601844511260235,
[junit4] 2> "nodeNames":["127.0.0.1:10002_solr"]}}
[junit4] 2> 1663301 DEBUG (simCloudManagerPool-1983-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1663302 DEBUG (simCloudManagerPool-1983-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 1663340 DEBUG (simCloudManagerPool-1983-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
[junit4] 2> 1663340 DEBUG (simCloudManagerPool-1983-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10002_solr]
[junit4] 2> 1663340 DEBUG (simCloudManagerPool-1983-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=3, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 1663340 DEBUG (AutoscalingActionExecutor-1985-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1663340 DEBUG (AutoscalingActionExecutor-1985-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 1663341 DEBUG (AutoscalingActionExecutor-1985-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 2975 ms for event id=5b0ddc4dc3879T4z9bvk1oicw7qrdzyny3ix5n8
[junit4] 2> 1663341 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 1663341 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 1663404 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTrigger
[junit4] 2> 1663405 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 1663405 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 1663405 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 1663405 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 1663405 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 1663405 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 1663405 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 1663406 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 1663406 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 1663406 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 1663406 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 1663407 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 1663407 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 1663407 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 1663407 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 1663408 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger
[junit4] 2> 1663425 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 1663425 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 1663441 DEBUG (ScheduledTrigger-1984-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1663441 DEBUG (ScheduledTrigger-1984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 1663441 DEBUG (ScheduledTrigger-1984-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1663443 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTriggerRestoreState
[junit4] 2> 1663444 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 1663444 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1663444 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1663466 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1663479 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1663479 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 1663479 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663479 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
[junit4] 2> 1663479 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 1663494 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1663494 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 1663494 DEBUG (ScheduledTrigger-1987-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663526 DEBUG (ScheduledTrigger-1987-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663546 DEBUG (ScheduledTrigger-1987-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663567 DEBUG (ScheduledTrigger-1987-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663579 DEBUG (simCloudManagerPool-1986-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1663580 DEBUG (simCloudManagerPool-1986-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 1663580 DEBUG (simCloudManagerPool-1986-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663580 DEBUG (simCloudManagerPool-1986-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 1663580 DEBUG (simCloudManagerPool-1986-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663580 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 1663581 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1663581 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 1663581 DEBUG (ScheduledTrigger-1987-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 1663581 DEBUG (ScheduledTrigger-1987-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1663587 DEBUG (ScheduledTrigger-1987-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1663601 DEBUG (ScheduledTrigger-1987-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 1663601 DEBUG (ScheduledTrigger-1987-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1663601 DEBUG (ScheduledTrigger-1987-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10003_solr at time 1601860564889885
[junit4] 2> 1663608 DEBUG (ScheduledTrigger-1987-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 1663615 DEBUG (simCloudManagerPool-1986-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1663615 DEBUG (simCloudManagerPool-1986-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 1663615 DEBUG (simCloudManagerPool-1986-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10003_solr]
[junit4] 2> 1663615 DEBUG (simCloudManagerPool-1986-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 1663615 DEBUG (simCloudManagerPool-1986-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr, 127.0.0.1:10003_solr]
[junit4] 2> 1663615 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 1663616 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1663616 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10003_solr
[junit4] 2> 1663616 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 1663616 DEBUG (ScheduledTrigger-1987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 1663616 DEBUG (ScheduledTrigger-1987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1663616 DEBUG (ScheduledTrigger-1987-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger firing registered processor for nodes: [127.0.0.1:10003_solr] added at times [1601861306680035], now=1601861316000585
[junit4] 2> 1663617 DEBUG (ScheduledTrigger-1987-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"5b0e2662a02e3T4z9bvk1oicw7qrdzyny3ix5nb",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":1601861306680035,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1601861306680035],
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 1663617 DEBUG (ScheduledTrigger-1987-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_restore_trigger]
[junit4] 2> 1663618 DEBUG (AutoscalingActionExecutor-1988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"5b0e2662a02e3T4z9bvk1oicw7qrdzyny3ix5nb",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":1601861306680035,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1601861306680035],
[junit4] 2> "_enqueue_time_":1601861385236135,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 1663619 DEBUG (AutoscalingActionExecutor-1988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1663619 DEBUG (AutoscalingActionExecutor-1988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_restore_trigger after 100ms
[junit4] 2> 1663619 DEBUG (AutoscalingActionExecutor-1988-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 25 ms for event id=5b0e2662a02e3T4z9bvk1oicw7qrdzyny3ix5nb
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTriggerRestoreState
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 2
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_restore_trigger
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 1663619 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 1663674 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
[junit4] 2> 1663674 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 1663674 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1663675 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1663676 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1663676 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 1663676 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663676 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion -1
[junit4] 2> 1663676 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 1663676 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1663676 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 1663676 DEBUG (ScheduledTrigger-1990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663689 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1663697 DEBUG (ScheduledTrigger-1990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663717 DEBUG (ScheduledTrigger-1990-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663737 DEBUG (ScheduledTrigger-1990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663758 DEBUG (ScheduledTrigger-1990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663786 DEBUG (ScheduledTrigger-1990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663790 DEBUG (simCloudManagerPool-1989-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1663790 DEBUG (simCloudManagerPool-1989-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 1663790 DEBUG (simCloudManagerPool-1989-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663790 INFO (simCloudManagerPool-1989-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 1663791 DEBUG (simCloudManagerPool-1989-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663791 DEBUG (simCloudManagerPool-1989-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 1663823 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 1663823 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1663823 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 1663824 DEBUG (ScheduledTrigger-1990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1663824 DEBUG (ScheduledTrigger-1990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1663824 DEBUG (ScheduledTrigger-1990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10004_solr at time 1601871695987785
[junit4] 2> 1663824 DEBUG (ScheduledTrigger-1990-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663844 DEBUG (ScheduledTrigger-1990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1663844 DEBUG (ScheduledTrigger-1990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 1663844 DEBUG (ScheduledTrigger-1990-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10004_solr] added at times [1601871695987785], now=1601872719741235
[junit4] 2> 1663844 DEBUG (ScheduledTrigger-1990-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"5b0e4d16a4449T4z9bvk1oicw7qrdzyny3ix5ne",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":1601871695987785,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1601871695987785],
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 1663845 DEBUG (ScheduledTrigger-1990-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 1663845 DEBUG (AutoscalingActionExecutor-1991-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"5b0e4d16a4449T4z9bvk1oicw7qrdzyny3ix5ne",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":1601871695987785,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1601871695987785],
[junit4] 2> "_enqueue_time_":1601872742163035,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 1663845 INFO (AutoscalingActionExecutor-1991-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"5b0e4d16a4449T4z9bvk1oicw7qrdzyny3ix5ne",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":1601871695987785,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1601871695987785],
[junit4] 2> "_enqueue_time_":1601872742163035,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 1663846 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1663846 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1663846 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1663846 WARN (AutoscalingActionExecutor-1991-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_trigger1 has been closed.
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$ScheduledTrigger.dequeue(ScheduledTriggers.java:522)
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:307)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1663846 DEBUG (AutoscalingActionExecutor-1991-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 49 ms for event id=5b0e4d16a4449T4z9bvk1oicw7qrdzyny3ix5ne
[junit4] 2> 1663847 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 1663847 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663847 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 1663847 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663847 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 1663847 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion -1
[junit4] 2> 1663847 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 1663847 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10001_solr
[junit4] 2> 1663847 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 1663848 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 1663848 DEBUG (ScheduledTrigger-1993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663848 DEBUG (ScheduledTrigger-1993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10001_solr
[junit4] 2> 1663848 DEBUG (ScheduledTrigger-1993-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"5b0e4d16a4449T4z9bvk1oicw7qrdzyny3ix5ne",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":1601871695987785,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1601871695987785],
[junit4] 2> "_dequeue_time_":1601872898529635,
[junit4] 2> "_enqueue_time_":1601872742163035,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 1663848 DEBUG (ScheduledTrigger-1993-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger1]
[junit4] 2> 1663849 DEBUG (AutoscalingActionExecutor-1994-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"5b0e4d16a4449T4z9bvk1oicw7qrdzyny3ix5ne",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":1601871695987785,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1601871695987785],
[junit4] 2> "_dequeue_time_":1601872898529635,
[junit4] 2> "_enqueue_time_":1601872742163035,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 1663849 DEBUG (ScheduledTrigger-1993-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1663849 INFO (AutoscalingActionExecutor-1994-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration -- event: {
[junit4] 2> "id":"5b0e4d16a4449T4z9bvk1oicw7qrdzyny3ix5ne",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":1601871695987785,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1601871695987785],
[junit4] 2> "_dequeue_time_":1601872898529635,
[junit4] 2> "_enqueue_time_":1601872742163035,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"],
[junit4] 2> "replaying":true}}
[junit4] 2> 1663849 DEBUG (ScheduledTrigger-1993-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1663850 DEBUG (AutoscalingActionExecutor-1994-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 1663850 DEBUG (AutoscalingActionExecutor-1994-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger1 after 100ms
[junit4] 2> 1663850 DEBUG (AutoscalingActionExecutor-1994-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 91 ms for event id=5b0e4d16a4449T4z9bvk1oicw7qrdzyny3ix5ne
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testEventQueue
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 2
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 1
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10001_solr
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 1663949 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 1663950 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 1663950 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 1
[junit4] 2> 1663950 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 3
[junit4] 2> 1663950 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 1663950 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_added_trigger1
[junit4] 2> 1663950 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 2
[junit4] 2> 1663950 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 1663950 DEBUG (ScheduledTrigger-1993-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663951 DEBUG (ScheduledTrigger-1993-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 1663952 DEBUG (ScheduledTrigger-1993-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 1663953 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
[junit4] 2> 1663953 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 1663953 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 1663954 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10001_solr
[junit4] 2> 1663954 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 1663954 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 1663954 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1663954 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:163)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1663954 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 1663955 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 1663955 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 1663955 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1663955 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion -1
[junit4] 2> 1663955 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 1663955 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1663955 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
[junit4] 2> 1663955 DEBUG (ScheduledTrigger-1996-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1663976 DEBUG (ScheduledTrigger-1996-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664006 DEBUG (ScheduledTrigger-1996-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664047 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664067 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664103 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664124 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664164 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664166 DEBUG (simCloudManagerPool-1995-thread-4) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for collection1 / shard1: {"core_node1":{
[junit4] 2> "node_name":"127.0.0.1:10004_solr",
[junit4] 2> "core":"collection1_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1"}}
[junit4] 2> 1664189 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664207 DEBUG (simCloudManagerPool-1995-thread-5) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1664207 DEBUG (simCloudManagerPool-1995-thread-5) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
[junit4] 2> 1664209 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664226 DEBUG (simCloudManagerPool-1995-thread-5) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1664229 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664233 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
[junit4] 2> 1664250 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664253 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1664253 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
[junit4] 2> 1664254 DEBUG (simCloudManagerPool-1995-thread-6) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 1664254 DEBUG (simCloudManagerPool-1995-thread-6) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
[junit4] 2> 1664254 DEBUG (simCloudManagerPool-1995-thread-6) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10004_solr, 127.0.0.1:10000_solr]
[junit4] 2> 1664270 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664286 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 15
[junit4] 2> 1664286 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 1664287 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 15, lastZnodeVersion 15
[junit4] 2> 1664307 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664339 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664359 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664392 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664420 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664450 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664477 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664501 DEBUG (ScheduledTrigger-1996-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664550 DEBUG (ScheduledTrigger-1996-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664571 DEBUG (ScheduledTrigger-1996-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 1664577 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"5b0ead71cd397T4z9bvk1oicw7qrdzyny3ix5ni",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1601897561379735,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10004_solr":250.0,
[junit4] 2> "127.0.0.1:10000_solr":250.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10004_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}}
[junit4] 2> 1664577 DEBUG (ScheduledTrigger-1996-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, search_rate_trigger]
[junit4] 2> 1664591 DEBUG (AutoscalingActionExecutor-1997-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"5b0ead71cd397T4z9bvk1oicw7qrdzyny3ix5ni",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1601897561379735,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10004_solr":250.0,
[junit4] 2> "127.0.0.1:10000_solr":250.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10004_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "_enqueue_time_":1601909378813835,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}}
[junit4] 2> 1664591 DEBUG (AutoscalingActionExecutor-1997-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"5b0ead71cd397T4z9bvk1oicw7qrdzyny3ix5ni",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1601897561379735,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10004_solr":250.0,
[junit4] 2> "127.0.0.1:10000_solr":250.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10004_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "_enqueue_time_":1601909378813835,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}} with context properties: {BEFORE_ACTION=[compute]}
[junit4] 2> 1664592 INFO (AutoscalingActionExecutor-1997-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10000_solr&type=NRT
[junit4] 2> 1664593 INFO (AutoscalingActionExecutor-1997-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10004_solr&type=NRT
[junit4] 2> 1664593 INFO (AutoscalingActionExecutor-1997-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10004_solr&type=NRT
[junit4] 2> 1664607 INFO (AutoscalingActionExecutor-1997-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 1601910877 sessionWrapper.createTime 1601910112751285, this.sessionWrapper.createTime 1601910112751285
[junit4] 2> 1664608 DEBUG (AutoscalingActionExecutor-1997-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"5b0ead71cd397T4z9bvk1oicw7qrdzyny3ix5ni",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1601897561379735,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10004_solr":250.0,
[junit4] 2> "127.0.0.1:10000_solr":250.0},
[junit4] 2> "__start__":1,
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10004_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10000_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "_enqueue_time_":1601909378813835,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@4210ee12, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@1f78789f, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@7318cbf5], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
[junit4] 2> 1664609 DEBUG (AutoscalingActionExecutor-1997-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10000_solr&type=NRT
[junit4] 2> 1664620 INFO (simCloudManagerPool-1995-thread-8) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10000_solr for creating new replica
[junit4] 2> 1664633 DEBUG (AutoscalingActionExecutor-1997-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10004_solr&type=NRT
[junit4] 2> 1664647 INFO (simCloudManagerPool-1995-thread-12) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10004_solr for creating new replica
[junit4] 2> 1664661 DEBUG (AutoscalingActionExecutor-1997-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10004_solr&type=NRT
[junit4] 2> 1664677 INFO (simCloudManagerPool-1995-thread-16) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10004_solr for creating new replica
[junit4] 2> 1664691 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSearchRate
[junit4] 2> 1664691 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 1664691 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 1664691 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 1664691 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 1664692 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 1664692 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 0
[junit4] 2> 1664692 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 1664734 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [collection1]
[junit4] 2> 1664734 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 2
[junit4] 2> 1664734 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[B8802F5058FD4B33]) [ ] o.a.s.c.a.s.SimSolrCloudT
[...truncated too long message...]
er Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 40880 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 40965 INFO (coreCloseExecutor-23-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7a70958c
[junit4] 2> 40984 INFO (coreCloseExecutor-23-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=2054198668
[junit4] 2> 41061 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@25b7f1a{/solr,null,UNAVAILABLE}
[junit4] 2> 41094 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 71102 WARN (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.e.j.u.t.QueuedThreadPool QueuedThreadPool@qtp572864418{STOPPING,8<=8<=10000,i=0,q=1} Couldn't stop Thread[qtp572864418-24,5,TGRP-TestExternalValueFeatures]
[junit4] 2> 71111 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.a.s.SolrTestCaseJ4 ###deleteCore
[junit4] 2> 71111 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1343444503
[junit4] 2> 71111 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 71152 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@126c7a6: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / null
[junit4] 2> 71152 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 71152 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@248fe3d0: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / null
[junit4] 2> 71153 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 71153 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@55b75394: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / null
[junit4] 2> 71155 INFO (coreCloseExecutor-25-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@3825bda
[junit4] 2> 71155 INFO (coreCloseExecutor-25-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=58874842
[junit4] 2> 71155 INFO (coreCloseExecutor-25-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4fefbb58: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / null
[junit4] 2> 71180 INFO (SUITE-TestExternalValueFeatures-seed#[BFB4FE6E10EE8A02]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 22, 2018 3:10:36 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 22, 2018 3:10:57 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestExternalValueFeatures:
[junit4] 2> 1) Thread[id=24, name=qtp572864418-24, state=TIMED_WAITING, group=TGRP-TestExternalValueFeatures]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 22, 2018 3:10:57 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=24, name=qtp572864418-24, state=TIMED_WAITING, group=TGRP-TestExternalValueFeatures]
[junit4] 2> Feb 22, 2018 3:11:00 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=24, name=qtp572864418-24, state=TIMED_WAITING, group=TGRP-TestExternalValueFeatures]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@50241075), locale=da, timezone=Asia/Ulan_Bator
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=2,free=219890752,total=342884352
[junit4] 2> NOTE: All tests run in this JVM: [TestExternalValueFeatures]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestExternalValueFeatures -Dtests.seed=BFB4FE6E10EE8A02 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=da -Dtests.timezone=Asia/Ulan_Bator -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 0.00s J0 | TestExternalValueFeatures (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.ltr.feature.TestExternalValueFeatures:
[junit4] > 1) Thread[id=24, name=qtp572864418-24, state=TIMED_WAITING, group=TGRP-TestExternalValueFeatures]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([BFB4FE6E10EE8A02]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=24, name=qtp572864418-24, state=TIMED_WAITING, group=TGRP-TestExternalValueFeatures]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([BFB4FE6E10EE8A02]:0)
[junit4] Completed [30/36 (1!)] on J0 in 97.17s, 2 tests, 2 errors <<< FAILURES!
[...truncated 40921 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2355 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2355/
No tests ran.
Build Log:
[...truncated 130 lines...]
ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-Tests-master - Build # 2354 - Still unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2354/
5 tests failed.
FAILED: org.apache.solr.cloud.TestUtilizeNode.test
Error Message:
no replica should be present in 127.0.0.1:34279_solr
Stack Trace:
java.lang.AssertionError: no replica should be present in 127.0.0.1:34279_solr
at __randomizedtesting.SeedInfo.seed([232BF95961C8BCC:8A66804F38E0E634]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.TestUtilizeNode.test(TestUtilizeNode.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost
Error Message:
The operations computed by ComputePlanAction should not be null SolrClientNodeStateProvider.DEBUG{AFTER_ACTION=[compute_plan, null], BEFORE_ACTION=[compute_plan, null]}
Stack Trace:
java.lang.AssertionError: The operations computed by ComputePlanAction should not be null SolrClientNodeStateProvider.DEBUG{AFTER_ACTION=[compute_plan, null], BEFORE_ACTION=[compute_plan, null]}
at __randomizedtesting.SeedInfo.seed([232BF95961C8BCC:32F25E171E6E6A90]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.autoscaling.ComputePlanActionTest.testNodeWithMultipleReplicasLost(ComputePlanActionTest.java:291)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([232BF95961C8BCC:5F7AA11C59DA2D83]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testSearchRate(TestLargeCluster.java:547)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.TestLBHttpSolrClient
Error Message:
1 thread leaked from SUITE scope at org.apache.solr.client.solrj.TestLBHttpSolrClient: 1) Thread[id=711, name=qtp806744391-711, state=TIMED_WAITING, group=TGRP-TestLBHttpSolrClient] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.TestLBHttpSolrClient:
1) Thread[id=711, name=qtp806744391-711, state=TIMED_WAITING, group=TGRP-TestLBHttpSolrClient]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([CFA3DBE1EAE1E10A]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.client.solrj.TestLBHttpSolrClient
Error Message:
There are still zombie threads that couldn't be terminated: 1) Thread[id=711, name=qtp806744391-711, state=TIMED_WAITING, group=TGRP-TestLBHttpSolrClient] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
1) Thread[id=711, name=qtp806744391-711, state=TIMED_WAITING, group=TGRP-TestLBHttpSolrClient]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
at __randomizedtesting.SeedInfo.seed([CFA3DBE1EAE1E10A]:0)
Build Log:
[...truncated 13214 lines...]
[junit4] Suite: org.apache.solr.cloud.TestUtilizeNode
[junit4] 2> 2378867 INFO (SUITE-TestUtilizeNode-seed#[232BF95961C8BCC]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/init-core-data-001
[junit4] 2> 2378867 WARN (SUITE-TestUtilizeNode-seed#[232BF95961C8BCC]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 2378867 INFO (SUITE-TestUtilizeNode-seed#[232BF95961C8BCC]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2378885 INFO (SUITE-TestUtilizeNode-seed#[232BF95961C8BCC]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2378886 INFO (SUITE-TestUtilizeNode-seed#[232BF95961C8BCC]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001
[junit4] 2> 2378886 INFO (SUITE-TestUtilizeNode-seed#[232BF95961C8BCC]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2378886 INFO (Thread-2167) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2378886 INFO (Thread-2167) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2378917 ERROR (Thread-2167) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2378986 INFO (SUITE-TestUtilizeNode-seed#[232BF95961C8BCC]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:36976
[junit4] 2> 2379005 INFO (zkConnectionManagerCallback-2283-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379020 INFO (jetty-launcher-2280-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2379020 INFO (jetty-launcher-2280-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2379020 INFO (jetty-launcher-2280-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2379020 INFO (jetty-launcher-2280-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2379021 INFO (jetty-launcher-2280-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2379021 INFO (jetty-launcher-2280-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2379021 INFO (jetty-launcher-2280-thread-3) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2379021 INFO (jetty-launcher-2280-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3ae92231{/solr,null,AVAILABLE}
[junit4] 2> 2379022 INFO (jetty-launcher-2280-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@53f92a09{HTTP/1.1,[http/1.1]}{127.0.0.1:44302}
[junit4] 2> 2379022 INFO (jetty-launcher-2280-thread-3) [ ] o.e.j.s.Server Started @2389480ms
[junit4] 2> 2379022 INFO (jetty-launcher-2280-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44302}
[junit4] 2> 2379022 ERROR (jetty-launcher-2280-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2379023 INFO (jetty-launcher-2280-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2379023 INFO (jetty-launcher-2280-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2379023 INFO (jetty-launcher-2280-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2379023 INFO (jetty-launcher-2280-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-21T05:01:03.425Z
[junit4] 2> 2379024 INFO (zkConnectionManagerCallback-2285-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379024 INFO (jetty-launcher-2280-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2379044 INFO (jetty-launcher-2280-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2379044 INFO (jetty-launcher-2280-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2379044 INFO (jetty-launcher-2280-thread-4) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2379045 INFO (jetty-launcher-2280-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@573b85f9{/solr,null,AVAILABLE}
[junit4] 2> 2379045 INFO (jetty-launcher-2280-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@240856e6{HTTP/1.1,[http/1.1]}{127.0.0.1:37256}
[junit4] 2> 2379045 INFO (jetty-launcher-2280-thread-4) [ ] o.e.j.s.Server Started @2389503ms
[junit4] 2> 2379045 INFO (jetty-launcher-2280-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37256}
[junit4] 2> 2379045 ERROR (jetty-launcher-2280-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2379045 INFO (jetty-launcher-2280-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2379045 INFO (jetty-launcher-2280-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2379045 INFO (jetty-launcher-2280-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2379046 INFO (jetty-launcher-2280-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-21T05:01:03.448Z
[junit4] 2> 2379062 INFO (jetty-launcher-2280-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2379062 INFO (jetty-launcher-2280-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2379062 INFO (jetty-launcher-2280-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2379062 INFO (zkConnectionManagerCallback-2287-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379062 INFO (jetty-launcher-2280-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2379064 INFO (jetty-launcher-2280-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7eae02bb{/solr,null,AVAILABLE}
[junit4] 2> 2379176 INFO (jetty-launcher-2280-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@622b5567{HTTP/1.1,[http/1.1]}{127.0.0.1:40831}
[junit4] 2> 2379176 INFO (jetty-launcher-2280-thread-1) [ ] o.e.j.s.Server Started @2389634ms
[junit4] 2> 2379192 INFO (jetty-launcher-2280-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36976/solr
[junit4] 2> 2379193 INFO (jetty-launcher-2280-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40831}
[junit4] 2> 2379234 ERROR (jetty-launcher-2280-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2379234 INFO (jetty-launcher-2280-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2379234 INFO (jetty-launcher-2280-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2379234 INFO (jetty-launcher-2280-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2379234 INFO (jetty-launcher-2280-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-21T05:01:03.636Z
[junit4] 2> 2379205 INFO (jetty-launcher-2280-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2379235 INFO (jetty-launcher-2280-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2379235 INFO (jetty-launcher-2280-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2379255 INFO (jetty-launcher-2280-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3c0f46c5{/solr,null,AVAILABLE}
[junit4] 2> 2379255 INFO (jetty-launcher-2280-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7a35b2e7{HTTP/1.1,[http/1.1]}{127.0.0.1:39778}
[junit4] 2> 2379255 INFO (jetty-launcher-2280-thread-2) [ ] o.e.j.s.Server Started @2389713ms
[junit4] 2> 2379255 INFO (jetty-launcher-2280-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39778}
[junit4] 2> 2379255 ERROR (jetty-launcher-2280-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2379260 INFO (jetty-launcher-2280-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2379260 INFO (jetty-launcher-2280-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2379260 INFO (jetty-launcher-2280-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2379260 INFO (jetty-launcher-2280-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-21T05:01:03.662Z
[junit4] 2> 2379261 INFO (zkConnectionManagerCallback-2291-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379267 INFO (jetty-launcher-2280-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36976/solr
[junit4] 2> 2379268 INFO (zkConnectionManagerCallback-2299-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379297 INFO (zkConnectionManagerCallback-2293-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379297 INFO (zkConnectionManagerCallback-2295-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379297 INFO (jetty-launcher-2280-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2379315 INFO (zkConnectionManagerCallback-2303-thread-1-processing-n:127.0.0.1:37256_solr) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379315 INFO (zkConnectionManagerCallback-2301-thread-1-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379322 INFO (jetty-launcher-2280-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2379428 INFO (jetty-launcher-2280-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36976/solr
[junit4] 2> 2379445 INFO (jetty-launcher-2280-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36976/solr
[junit4] 2> 2379465 INFO (zkConnectionManagerCallback-2309-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379467 INFO (zkConnectionManagerCallback-2313-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379490 INFO (zkConnectionManagerCallback-2315-thread-1-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379490 INFO (zkConnectionManagerCallback-2317-thread-1-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2379520 INFO (jetty-launcher-2280-thread-3) [n:127.0.0.1:44302_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2379521 INFO (jetty-launcher-2280-thread-3) [n:127.0.0.1:44302_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44302_solr
[junit4] 2> 2379538 INFO (jetty-launcher-2280-thread-3) [n:127.0.0.1:44302_solr ] o.a.s.c.Overseer Overseer (id=72157645504643080-127.0.0.1:44302_solr-n_0000000000) starting
[junit4] 2> 2379541 INFO (jetty-launcher-2280-thread-1) [n:127.0.0.1:40831_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2379554 INFO (jetty-launcher-2280-thread-1) [n:127.0.0.1:40831_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40831_solr
[junit4] 2> 2379583 INFO (zkCallback-2316-thread-1-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2379601 INFO (jetty-launcher-2280-thread-2) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2379603 INFO (jetty-launcher-2280-thread-2) [n:127.0.0.1:39778_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2379620 INFO (zkCallback-2300-thread-1-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2379637 INFO (jetty-launcher-2280-thread-3) [n:127.0.0.1:44302_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44302_solr
[junit4] 2> 2379638 INFO (jetty-launcher-2280-thread-2) [n:127.0.0.1:39778_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39778_solr
[junit4] 2> 2379668 INFO (zkCallback-2300-thread-2-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2379680 INFO (zkCallback-2316-thread-1-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 2379680 INFO (zkCallback-2314-thread-1-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 2379705 INFO (zkCallback-2300-thread-1-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2379813 INFO (jetty-launcher-2280-thread-1) [n:127.0.0.1:40831_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40831.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2379916 INFO (jetty-launcher-2280-thread-1) [n:127.0.0.1:40831_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40831.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2379917 INFO (jetty-launcher-2280-thread-1) [n:127.0.0.1:40831_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40831.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2379918 INFO (jetty-launcher-2280-thread-1) [n:127.0.0.1:40831_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node1/.
[junit4] 2> 2380042 INFO (jetty-launcher-2280-thread-4) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2380076 INFO (jetty-launcher-2280-thread-4) [n:127.0.0.1:37256_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2380078 INFO (jetty-launcher-2280-thread-4) [n:127.0.0.1:37256_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37256_solr
[junit4] 2> 2380103 INFO (zkCallback-2316-thread-1-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2380105 INFO (jetty-launcher-2280-thread-2) [n:127.0.0.1:39778_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39778.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2380111 INFO (zkCallback-2300-thread-2-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2380141 INFO (jetty-launcher-2280-thread-3) [n:127.0.0.1:44302_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44302.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2380156 INFO (zkCallback-2314-thread-1-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2380195 INFO (jetty-launcher-2280-thread-2) [n:127.0.0.1:39778_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39778.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2380195 INFO (jetty-launcher-2280-thread-2) [n:127.0.0.1:39778_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39778.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2380209 INFO (zkCallback-2302-thread-1-processing-n:127.0.0.1:37256_solr) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2380230 INFO (jetty-launcher-2280-thread-3) [n:127.0.0.1:44302_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44302.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2380230 INFO (jetty-launcher-2280-thread-3) [n:127.0.0.1:44302_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44302.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2380231 INFO (jetty-launcher-2280-thread-3) [n:127.0.0.1:44302_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node3/.
[junit4] 2> 2380249 INFO (jetty-launcher-2280-thread-2) [n:127.0.0.1:39778_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node2/.
[junit4] 2> 2380718 INFO (jetty-launcher-2280-thread-4) [n:127.0.0.1:37256_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37256.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2380757 INFO (jetty-launcher-2280-thread-4) [n:127.0.0.1:37256_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37256.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2380790 INFO (jetty-launcher-2280-thread-4) [n:127.0.0.1:37256_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37256.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2380792 INFO (jetty-launcher-2280-thread-4) [n:127.0.0.1:37256_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node4/.
[junit4] 2> 2381088 INFO (zkConnectionManagerCallback-2325-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2381136 INFO (zkConnectionManagerCallback-2329-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2381218 INFO (SUITE-TestUtilizeNode-seed#[232BF95961C8BCC]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2381218 INFO (SUITE-TestUtilizeNode-seed#[232BF95961C8BCC]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36976/solr ready
[junit4] 2> 2381327 INFO (qtp1652280324-10169) [n:127.0.0.1:40831_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2381585 INFO (qtp1652280324-10169) [n:127.0.0.1:40831_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=257
[junit4] 2> 2382474 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 2382579 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2382700 INFO (zkConnectionManagerCallback-2332-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2382707 INFO (qtp1652280324-10194) [n:127.0.0.1:40831_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&name=utilizenodecoll&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2382726 INFO (OverseerThreadFactory-3333-thread-2-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.a.c.CreateCollectionCmd Create collection utilizenodecoll
[junit4] 2> 2382726 INFO (OverseerCollectionConfigSetProcessor-72157645504643080-127.0.0.1:44302_solr-n_0000000000) [n:127.0.0.1:44302_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2382967 INFO (OverseerStateUpdate-72157645504643080-127.0.0.1:44302_solr-n_0000000000) [n:127.0.0.1:44302_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"utilizenodecoll_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:39778/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2383053 INFO (OverseerStateUpdate-72157645504643080-127.0.0.1:44302_solr-n_0000000000) [n:127.0.0.1:44302_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"utilizenodecoll_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:40831/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2383066 INFO (OverseerStateUpdate-72157645504643080-127.0.0.1:44302_solr-n_0000000000) [n:127.0.0.1:44302_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"utilizenodecoll_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44302/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2383067 INFO (OverseerStateUpdate-72157645504643080-127.0.0.1:44302_solr-n_0000000000) [n:127.0.0.1:44302_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"utilizenodecoll",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"utilizenodecoll_shard2_replica_n5",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:37256/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2383272 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard1_replica_n1&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2383347 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2383367 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard2_replica_n4&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2383368 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2383385 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard1_replica_n2&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2383386 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2383386 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard2_replica_n5&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2383399 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2383598 INFO (zkCallback-2314-thread-1-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2383599 INFO (zkCallback-2316-thread-1-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2383599 INFO (zkCallback-2300-thread-2-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2383599 INFO (zkCallback-2300-thread-1-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2383605 INFO (zkCallback-2302-thread-1-processing-n:127.0.0.1:37256_solr) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2383614 INFO (zkCallback-2314-thread-2-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2383616 INFO (zkCallback-2302-thread-2-processing-n:127.0.0.1:37256_solr) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2383616 INFO (zkCallback-2316-thread-2-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2384554 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2384563 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2384578 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2384582 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2384600 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.s.IndexSchema [utilizenodecoll_shard1_replica_n2] Schema name=cloud-dynamic
[junit4] 2> 2384675 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.s.IndexSchema [utilizenodecoll_shard2_replica_n5] Schema name=cloud-dynamic
[junit4] 2> 2384686 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.s.IndexSchema [utilizenodecoll_shard2_replica_n4] Schema name=cloud-dynamic
[junit4] 2> 2384695 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.s.IndexSchema [utilizenodecoll_shard1_replica_n1] Schema name=cloud-dynamic
[junit4] 2> 2384742 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2384742 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'utilizenodecoll_shard1_replica_n2' using configuration from collection utilizenodecoll, trusted=true
[junit4] 2> 2384743 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40831.solr.core.utilizenodecoll.shard1.replica_n2' (registry 'solr.core.utilizenodecoll.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2384743 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2384743 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.c.SolrCore [[utilizenodecoll_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node1/utilizenodecoll_shard1_replica_n2], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node1/./utilizenodecoll_shard1_replica_n2/data/]
[junit4] 2> 2384965 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2384965 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'utilizenodecoll_shard1_replica_n1' using configuration from collection utilizenodecoll, trusted=true
[junit4] 2> 2384966 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39778.solr.core.utilizenodecoll.shard1.replica_n1' (registry 'solr.core.utilizenodecoll.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2384966 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2384966 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SolrCore [[utilizenodecoll_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node2/utilizenodecoll_shard1_replica_n1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node2/./utilizenodecoll_shard1_replica_n1/data/]
[junit4] 2> 2385057 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2385057 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'utilizenodecoll_shard2_replica_n5' using configuration from collection utilizenodecoll, trusted=true
[junit4] 2> 2385057 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37256.solr.core.utilizenodecoll.shard2.replica_n5' (registry 'solr.core.utilizenodecoll.shard2.replica_n5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2385073 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2385074 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.c.SolrCore [[utilizenodecoll_shard2_replica_n5] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node4/utilizenodecoll_shard2_replica_n5], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node4/./utilizenodecoll_shard2_replica_n5/data/]
[junit4] 2> 2385109 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema cloud-dynamic/1.6 with uniqueid field id
[junit4] 2> 2385109 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'utilizenodecoll_shard2_replica_n4' using configuration from collection utilizenodecoll, trusted=true
[junit4] 2> 2385110 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44302.solr.core.utilizenodecoll.shard2.replica_n4' (registry 'solr.core.utilizenodecoll.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2385110 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2385110 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SolrCore [[utilizenodecoll_shard2_replica_n4] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node3/utilizenodecoll_shard2_replica_n4], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J1/temp/solr.cloud.TestUtilizeNode_232BF95961C8BCC-001/tempDir-001/node3/./utilizenodecoll_shard2_replica_n4/data/]
[junit4] 2> 2385330 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2385330 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2385331 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2385331 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2385332 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.s.SolrIndexSearcher Opening [Searcher@5201432c[utilizenodecoll_shard2_replica_n4] main]
[junit4] 2> 2385362 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2385362 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2385362 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2385363 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592985407733104640
[junit4] 2> 2385378 INFO (searcherExecutor-3345-thread-1-processing-n:127.0.0.1:44302_solr x:utilizenodecoll_shard2_replica_n4 s:shard2 c:utilizenodecoll r:core_node7) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SolrCore [utilizenodecoll_shard2_replica_n4] Registered new searcher Searcher@5201432c[utilizenodecoll_shard2_replica_n4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2385395 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2385395 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2385396 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2385396 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2385398 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.s.SolrIndexSearcher Opening [Searcher@30f23516[utilizenodecoll_shard2_replica_n5] main]
[junit4] 2> 2385416 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2385416 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2385416 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2385417 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2385418 INFO (searcherExecutor-3344-thread-1-processing-n:127.0.0.1:37256_solr x:utilizenodecoll_shard2_replica_n5 s:shard2 c:utilizenodecoll r:core_node8) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.c.SolrCore [utilizenodecoll_shard2_replica_n5] Registered new searcher Searcher@30f23516[utilizenodecoll_shard2_replica_n5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2385418 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592985407790776320
[junit4] 2> 2385435 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2385435 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2385437 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2385437 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2385451 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@419e63a0[utilizenodecoll_shard1_replica_n1] main]
[junit4] 2> 2385452 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2385453 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2385453 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2385453 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592985407827476480
[junit4] 2> 2385454 INFO (searcherExecutor-3343-thread-1-processing-n:127.0.0.1:39778_solr x:utilizenodecoll_shard1_replica_n1 s:shard1 c:utilizenodecoll r:core_node3) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SolrCore [utilizenodecoll_shard1_replica_n1] Registered new searcher Searcher@419e63a0[utilizenodecoll_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2385477 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2385477 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2385478 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2385478 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2385485 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2385504 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@794ef231[utilizenodecoll_shard1_replica_n2] main]
[junit4] 2> 2385505 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2385505 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2385506 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2385506 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1592985407883051008
[junit4] 2> 2385507 INFO (searcherExecutor-3342-thread-1-processing-n:127.0.0.1:40831_solr x:utilizenodecoll_shard1_replica_n2 s:shard1 c:utilizenodecoll r:core_node6) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.c.SolrCore [utilizenodecoll_shard1_replica_n2] Registered new searcher Searcher@794ef231[utilizenodecoll_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2385589 INFO (zkCallback-2302-thread-2-processing-n:127.0.0.1:37256_solr) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2385589 INFO (zkCallback-2314-thread-1-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2385589 INFO (zkCallback-2302-thread-1-processing-n:127.0.0.1:37256_solr) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2385590 INFO (zkCallback-2300-thread-3-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2385590 INFO (zkCallback-2300-thread-1-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2385590 INFO (zkCallback-2316-thread-1-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2385590 INFO (zkCallback-2316-thread-2-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2385590 INFO (zkCallback-2314-thread-2-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2385916 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2385916 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2385917 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44302/solr/utilizenodecoll_shard2_replica_n4/
[junit4] 2> 2385917 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.PeerSync PeerSync: core=utilizenodecoll_shard2_replica_n4 url=http://127.0.0.1:44302/solr START replicas=[http://127.0.0.1:37256/solr/utilizenodecoll_shard2_replica_n5/] nUpdates=100
[junit4] 2> 2385924 INFO (qtp1484338554-10189) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.c.S.Request [utilizenodecoll_shard2_replica_n5] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 2385986 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2385986 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2385986 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39778/solr/utilizenodecoll_shard1_replica_n1/
[junit4] 2> 2385986 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=utilizenodecoll_shard1_replica_n1 url=http://127.0.0.1:39778/solr START replicas=[http://127.0.0.1:40831/solr/utilizenodecoll_shard1_replica_n2/] nUpdates=100
[junit4] 2> 2385990 INFO (qtp1652280324-10196) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.c.S.Request [utilizenodecoll_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2386217 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.u.PeerSync PeerSync: core=utilizenodecoll_shard2_replica_n4 url=http://127.0.0.1:44302/solr DONE. We have no versions. sync failed.
[junit4] 2> 2386218 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2386218 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2386218 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2386221 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44302/solr/utilizenodecoll_shard2_replica_n4/ shard2
[junit4] 2> 2386287 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=utilizenodecoll_shard1_replica_n1 url=http://127.0.0.1:39778/solr DONE. We have no versions. sync failed.
[junit4] 2> 2386287 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2386287 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2386287 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2386290 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39778/solr/utilizenodecoll_shard1_replica_n1/ shard1
[junit4] 2> 2386391 INFO (zkCallback-2302-thread-1-processing-n:127.0.0.1:37256_solr) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386391 INFO (zkCallback-2316-thread-1-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386391 INFO (zkCallback-2302-thread-2-processing-n:127.0.0.1:37256_solr) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386391 INFO (zkCallback-2314-thread-1-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386391 INFO (zkCallback-2300-thread-3-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386391 INFO (zkCallback-2316-thread-2-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386391 INFO (zkCallback-2300-thread-1-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386391 INFO (zkCallback-2314-thread-2-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386422 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2386424 INFO (qtp1775104451-10171) [n:127.0.0.1:44302_solr c:utilizenodecoll s:shard2 r:core_node7 x:utilizenodecoll_shard2_replica_n4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard2_replica_n4&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3073
[junit4] 2> 2386441 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2386442 INFO (qtp163816418-10168) [n:127.0.0.1:39778_solr c:utilizenodecoll s:shard1 r:core_node3 x:utilizenodecoll_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard1_replica_n1&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3170
[junit4] 2> 2386469 INFO (qtp1484338554-10182) [n:127.0.0.1:37256_solr c:utilizenodecoll s:shard2 r:core_node8 x:utilizenodecoll_shard2_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard2_replica_n5&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3083
[junit4] 2> 2386534 INFO (qtp1652280324-10195) [n:127.0.0.1:40831_solr c:utilizenodecoll s:shard1 r:core_node6 x:utilizenodecoll_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=conf1&newCollection=true&name=utilizenodecoll_shard1_replica_n2&action=CREATE&numShards=2&collection=utilizenodecoll&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=3148
[junit4] 2> 2386541 INFO (qtp1652280324-10194) [n:127.0.0.1:40831_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2386638 INFO (zkCallback-2302-thread-2-processing-n:127.0.0.1:37256_solr) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386638 INFO (zkCallback-2316-thread-1-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386638 INFO (zkCallback-2302-thread-1-processing-n:127.0.0.1:37256_solr) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386638 INFO (zkCallback-2314-thread-2-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386638 INFO (zkCallback-2316-thread-2-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386638 INFO (zkCallback-2314-thread-1-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386639 INFO (zkCallback-2300-thread-1-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386639 INFO (zkCallback-2300-thread-3-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/utilizenodecoll/state.json] for collection [utilizenodecoll] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2386740 INFO (OverseerCollectionConfigSetProcessor-72157645504643080-127.0.0.1:44302_solr-n_0000000000) [n:127.0.0.1:44302_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2387551 INFO (qtp1652280324-10194) [n:127.0.0.1:40831_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf1&name=utilizenodecoll&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2} status=0 QTime=4844
[junit4] 2> 2387564 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2387601 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2387601 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2387601 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2387602 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1f12e52f{/solr,null,AVAILABLE}
[junit4] 2> 2387602 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@137f9c4b{HTTP/1.1,[http/1.1]}{127.0.0.1:34279}
[junit4] 2> 2387602 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.e.j.s.Server Started @2398060ms
[junit4] 2> 2387602 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34279}
[junit4] 2> 2387602 ERROR (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2387603 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2387603 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2387603 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2387603 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-02-21T05:01:12.005Z
[junit4] 2> 2387710 INFO (zkConnectionManagerCallback-2334-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2387730 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2387801 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36976/solr
[junit4] 2> 2387853 INFO (zkConnectionManagerCallback-2338-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2387877 INFO (zkConnectionManagerCallback-2340-thread-1-processing-n:127.0.0.1:34279_solr) [n:127.0.0.1:34279_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2387927 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [n:127.0.0.1:34279_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2387928 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [n:127.0.0.1:34279_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2387942 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [n:127.0.0.1:34279_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34279_solr
[junit4] 2> 2387962 INFO (zkCallback-2302-thread-1-processing-n:127.0.0.1:37256_solr) [n:127.0.0.1:37256_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2387966 INFO (zkCallback-2314-thread-1-processing-n:127.0.0.1:39778_solr) [n:127.0.0.1:39778_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2387966 INFO (zkCallback-2316-thread-1-processing-n:127.0.0.1:40831_solr) [n:127.0.0.1:40831_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2387966 INFO (zkCallback-2328-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2387967 INFO (zkCallback-2328-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2387978 INFO (zkCallback-2300-thread-3-processing-n:127.0.0.1:44302_solr) [n:127.0.0.1:44302_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2388011 INFO (zkCallback-2339-thread-1-processing-n:127.0.0.1:34279_solr) [n:127.0.0.1:34279_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2388445 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [n:127.0.0.1:34279_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34279.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2388500 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [n:127.0.0.1:34279_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34279.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@678644ba
[junit4] 2> 2388500 INFO (TEST-TestUtilizeNode.test-seed#[232BF95961C8BCC]) [n:127.0.0.1:34279_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34279.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbea
[...truncated too long message...]
EAE1E10A]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1244992770
[junit4] 2> 68077 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[CFA3DBE1EAE1E10A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 68077 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[CFA3DBE1EAE1E10A]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@30edef01: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@4219d5b5
[junit4] 2> 68079 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[CFA3DBE1EAE1E10A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 68079 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[CFA3DBE1EAE1E10A]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@78934828: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@363f6c28
[junit4] 2> 68079 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[CFA3DBE1EAE1E10A]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 68079 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[CFA3DBE1EAE1E10A]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@ed6f823: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@c84fdc1
[junit4] 2> 68101 INFO (coreCloseExecutor-301-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@13780780
[junit4] 2> 68101 INFO (coreCloseExecutor-301-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=326633344
[junit4] 2> 68101 INFO (coreCloseExecutor-301-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5873f584: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@695e16c2
[junit4] 2> 68186 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[CFA3DBE1EAE1E10A]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4128e52d{/solr,null,UNAVAILABLE}
[junit4] 2> 68186 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[CFA3DBE1EAE1E10A]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 98199 WARN (TEST-TestLBHttpSolrClient.testReliability-seed#[CFA3DBE1EAE1E10A]) [ ] o.e.j.u.t.QueuedThreadPool QueuedThreadPool@qtp806744391{STOPPING,8<=8<=10000,i=0,q=1} Couldn't stop Thread[qtp806744391-711,5,TGRP-TestLBHttpSolrClient]
[junit4] 2> 98201 INFO (TEST-TestLBHttpSolrClient.testReliability-seed#[CFA3DBE1EAE1E10A]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testReliability
[junit4] 2> 98215 INFO (SUITE-TestLBHttpSolrClient-seed#[CFA3DBE1EAE1E10A]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Feb 21, 2018 5:48:29 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> Feb 21, 2018 5:48:49 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.TestLBHttpSolrClient:
[junit4] 2> 1) Thread[id=711, name=qtp806744391-711, state=TIMED_WAITING, group=TGRP-TestLBHttpSolrClient]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Feb 21, 2018 5:48:49 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=711, name=qtp806744391-711, state=TIMED_WAITING, group=TGRP-TestLBHttpSolrClient]
[junit4] 2> Feb 21, 2018 5:48:53 AM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> SEVERE: There are still zombie threads that couldn't be terminated:
[junit4] 2> 1) Thread[id=711, name=qtp806744391-711, state=TIMED_WAITING, group=TGRP-TestLBHttpSolrClient]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {name=PostingsFormat(name=Direct), id=PostingsFormat(name=Direct)}, docValues:{}, maxPointsInLeafNode=2027, maxMBSortInHeap=6.2956608669212235, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@7d118aa), locale=zh-HK, timezone=PST8PDT
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=4,free=219802744,total=334495744
[junit4] 2> NOTE: All tests run in this JVM: [TestUpdateRequestCodec, FloorEvaluatorTest, HttpSolrClientSSLAuthConPoolTest, SolrPingTest, TestV2Request, UniformDistributionEvaluatorTest, CloudSolrClientMultiConstructorTest, AppendEvaluatorTest, SolrQueryTest, CollectionAdminRequestRequiredParamsTest, GraphTest, FacetFieldTest, StreamExpressionToExpessionTest, CeilingEvaluatorTest, SubtractEvaluatorTest, HyperbolicCosineEvaluatorTest, TestLBHttpSolrClient]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestLBHttpSolrClient -Dtests.seed=CFA3DBE1EAE1E10A -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=zh-HK -Dtests.timezone=PST8PDT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestLBHttpSolrClient (suite) <<<
[junit4] > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.client.solrj.TestLBHttpSolrClient:
[junit4] > 1) Thread[id=711, name=qtp806744391-711, state=TIMED_WAITING, group=TGRP-TestLBHttpSolrClient]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([CFA3DBE1EAE1E10A]:0)Throwable #2: com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
[junit4] > 1) Thread[id=711, name=qtp806744391-711, state=TIMED_WAITING, group=TGRP-TestLBHttpSolrClient]
[junit4] > at sun.misc.Unsafe.park(Native Method)
[junit4] > at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] > at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
[junit4] > at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] > at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > at __randomizedtesting.SeedInfo.seed([CFA3DBE1EAE1E10A]:0)
[junit4] Completed [34/146 (1!)] on J1 in 81.72s, 3 tests, 2 errors <<< FAILURES!
[...truncated 47835 lines...]
[JENKINS] Lucene-Solr-Tests-master - Build # 2353 - Still Failing
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-Tests-master/2353/
5 tests failed.
FAILED: org.apache.solr.handler.dataimport.TestContentStreamDataSource.testCommitWithin
Error Message:
expected:<0> but was:<2>
Stack Trace:
java.lang.AssertionError: expected:<0> but was:<2>
at __randomizedtesting.SeedInfo.seed([7C0C821FF129BF1A:C6DEED677207510F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.handler.dataimport.TestContentStreamDataSource.testCommitWithin(TestContentStreamDataSource.java:98)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.PeerSyncReplicationTest.test
Error Message:
expected:<0> but was:<1>
Stack Trace:
java.lang.AssertionError: expected:<0> but was:<1>
at __randomizedtesting.SeedInfo.seed([A83ABF3BE9CAEDB3:206E80E14736804B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.junit.Assert.assertEquals(Assert.java:456)
at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:203)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ReplaceNodeNoTargetTest.test
Error Message:
Stack Trace:
java.lang.AssertionError
at __randomizedtesting.SeedInfo.seed([A83ABF3BE9CAEDB3:206E80E14736804B]:0)
at org.junit.Assert.fail(Assert.java:92)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertTrue(Assert.java:54)
at org.apache.solr.cloud.ReplaceNodeNoTargetTest.test(ReplaceNodeNoTargetTest.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration
Error Message:
Path /autoscaling/nodeAdded/127.0.0.1:10003_solr should have been deleted
Stack Trace:
java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:10003_solr should have been deleted
at __randomizedtesting.SeedInfo.seed([A83ABF3BE9CAEDB3:B0803737E7FF205C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertFalse(Assert.java:68)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration(TestTriggerIntegration.java:844)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication
Error Message:
Index: 0, Size: 0
Stack Trace:
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at __randomizedtesting.SeedInfo.seed([A83ABF3BE9CAEDB3:BC72E46ECACD50AD]:0)
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication(TestReplicationHandler.java:561)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12188 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[junit4] 2> 448526 INFO (SUITE-TestTriggerIntegration-seed#[A83ABF3BE9CAEDB3]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_A83ABF3BE9CAEDB3-001/init-core-data-001
[junit4] 2> 448540 WARN (SUITE-TestTriggerIntegration-seed#[A83ABF3BE9CAEDB3]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=68 numCloses=68
[junit4] 2> 448540 INFO (SUITE-TestTriggerIntegration-seed#[A83ABF3BE9CAEDB3]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 448541 INFO (SUITE-TestTriggerIntegration-seed#[A83ABF3BE9CAEDB3]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 448590 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 448591 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 0
[junit4] 2> 448591 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 448591 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion -1
[junit4] 2> 448591 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 0
[junit4] 2> 448623 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 448623 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 0, lastZnodeVersion 0
[junit4] 2> 448670 DEBUG (ScheduledTrigger-440-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448677 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeMarkersRegistration
[junit4] 2> 448677 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 448677 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 448680 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 448712 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 448712 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 448713 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 448713 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:163)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 448714 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 448739 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 448739 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10000_solr]
[junit4] 2> 448739 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 448739 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 448740 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 448742 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 448742 DEBUG (ScheduledTrigger-443-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448762 DEBUG (ScheduledTrigger-443-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448783 DEBUG (ScheduledTrigger-443-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448803 DEBUG (ScheduledTrigger-443-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448814 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== KILL OVERSEER 1
[junit4] 2> 448814 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 448815 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 448815 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 448816 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 448816 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 448816 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion -1
[junit4] 2> 448816 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 448816 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10000_solr
[junit4] 2> 448816 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 448816 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 448836 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448836 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10000_solr
[junit4] 2> 448857 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448877 DEBUG (ScheduledTrigger-446-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448897 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448916 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== ADD TRIGGERS
[junit4] 2> 448918 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448950 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448970 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448991 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 448991 DEBUG (simCloudManagerPool-445-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 448992 DEBUG (simCloudManagerPool-445-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 448992 DEBUG (simCloudManagerPool-445-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 448994 DEBUG (simCloudManagerPool-445-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 448994 DEBUG (simCloudManagerPool-445-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventMarkerAction}], enabled=true}
[junit4] 2> 448994 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 448994 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 448995 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 448995 DEBUG (ScheduledTrigger-446-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 448995 DEBUG (ScheduledTrigger-446-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 449008 DEBUG (simCloudManagerPool-445-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 449008 DEBUG (simCloudManagerPool-445-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 449008 DEBUG (simCloudManagerPool-445-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 449008 DEBUG (simCloudManagerPool-445-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 449008 DEBUG (simCloudManagerPool-445-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 449008 DEBUG (simCloudManagerPool-445-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventMarkerAction}], enabled=true}
[junit4] 2> 449008 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 449009 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 449009 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 449009 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 449009 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== ADD NODE 1
[junit4] 2> 449011 DEBUG (ScheduledTrigger-446-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 449056 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 449057 DEBUG (ScheduledTrigger-446-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 449057 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 449057 DEBUG (ScheduledTrigger-446-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10003_solr at time 1483592627547931
[junit4] 2> 449076 DEBUG (ScheduledTrigger-446-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 449077 DEBUG (ScheduledTrigger-446-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 449077 DEBUG (ScheduledTrigger-446-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 449077 DEBUG (ScheduledTrigger-446-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 449077 DEBUG (ScheduledTrigger-446-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10003_solr] added at times [1483592627547931], now=1483593642521831
[junit4] 2> 449077 DEBUG (ScheduledTrigger-446-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"54551d38e6f1bT9ytwjybgp40jbpq7iexv6qgm8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":1483592627547931,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1483592627547931],
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> 449145 DEBUG (ScheduledTrigger-446-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger, node_added_trigger]
[junit4] 2> 449146 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeMarkersRegistration
[junit4] 2> 449146 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 449146 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 449146 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 449146 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 449146 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 3
[junit4] 2> 449146 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 3
[junit4] 2> 449146 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 1
[junit4] 2> 449146 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10000_solr
[junit4] 2> 449147 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 449147 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 449147 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 449147 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 449147 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 449148 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 449148 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 1
[junit4] 2> 449148 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 449184 DEBUG (AutoscalingActionExecutor-447-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"54551d38e6f1bT9ytwjybgp40jbpq7iexv6qgm8",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":1483592627547931,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1483592627547931],
[junit4] 2> "_enqueue_time_":1483597091059781,
[junit4] 2> "nodeNames":["127.0.0.1:10003_solr"]}}
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestTriggerIntegration -Dtests.method=testNodeMarkersRegistration -Dtests.seed=A83ABF3BE9CAEDB3 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.locale=en-IN -Dtests.timezone=Asia/Thimbu -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 0.59s J2 | TestTriggerIntegration.testNodeMarkersRegistration <<<
[junit4] > Throwable #1: java.lang.AssertionError: Path /autoscaling/nodeAdded/127.0.0.1:10003_solr should have been deleted
[junit4] > at __randomizedtesting.SeedInfo.seed([A83ABF3BE9CAEDB3:B0803737E7FF205C]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testNodeMarkersRegistration(TestTriggerIntegration.java:844)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 449230 DEBUG (AutoscalingActionExecutor-447-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 449230 DEBUG (AutoscalingActionExecutor-447-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
[junit4] 2> 449230 DEBUG (AutoscalingActionExecutor-447-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 449230 DEBUG (AutoscalingActionExecutor-447-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 2334 ms for event id=54551d38e6f1bT9ytwjybgp40jbpq7iexv6qgm8
[junit4] 2> 449257 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTriggerRestoreState
[junit4] 2> 449257 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 449257 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 5
[junit4] 2> 449257 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 449257 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 449257 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 5, lastZnodeVersion 5
[junit4] 2> 449258 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10000_solr
[junit4] 2> 449258 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 449258 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:175)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 449258 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 449272 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 449272 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 449272 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 449272 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
[junit4] 2> 449272 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 449282 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 449282 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 449282 DEBUG (ScheduledTrigger-449-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449302 DEBUG (ScheduledTrigger-449-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449323 DEBUG (ScheduledTrigger-449-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449343 DEBUG (ScheduledTrigger-449-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449361 DEBUG (simCloudManagerPool-448-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 449361 DEBUG (simCloudManagerPool-448-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 449385 DEBUG (ScheduledTrigger-449-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 449385 DEBUG (simCloudManagerPool-448-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10002_solr]
[junit4] 2> 449386 DEBUG (simCloudManagerPool-448-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10004_solr, 127.0.0.1:10002_solr]
[junit4] 2> 449386 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 449386 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 449386 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 449386 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 449386 DEBUG (ScheduledTrigger-449-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 449386 DEBUG (ScheduledTrigger-449-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10004_solr
[junit4] 2> 449406 DEBUG (ScheduledTrigger-449-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449406 DEBUG (ScheduledTrigger-449-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10004_solr
[junit4] 2> 449407 DEBUG (ScheduledTrigger-449-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 449417 DEBUG (simCloudManagerPool-448-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 449417 DEBUG (simCloudManagerPool-448-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 449417 DEBUG (simCloudManagerPool-448-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 449417 DEBUG (simCloudManagerPool-448-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 449417 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 449418 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 449418 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 449418 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 449418 DEBUG (ScheduledTrigger-449-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 449418 DEBUG (ScheduledTrigger-449-thread-4) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10004_solr]
[junit4] 2> 449418 DEBUG (ScheduledTrigger-449-thread-4) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"54555a9a920d9T9ytwjybgp40jbpq7iexv6qgmb",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":1483609104523481,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1483609104523481],
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 449447 DEBUG (ScheduledTrigger-449-thread-4) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_restore_trigger]
[junit4] 2> 449447 DEBUG (AutoscalingActionExecutor-450-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"54555a9a920d9T9ytwjybgp40jbpq7iexv6qgmb",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":1483609104523481,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1483609104523481],
[junit4] 2> "_enqueue_time_":1483612126218481,
[junit4] 2> "nodeNames":["127.0.0.1:10004_solr"]}}
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTriggerRestoreState
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 2
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 1
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## 127.0.0.1:10004_solr
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: []
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 0
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 0
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 0
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 1
[junit4] 2> 449448 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 449449 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: node_lost_restore_trigger
[junit4] 2> 449450 DEBUG (AutoscalingActionExecutor-450-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 449450 DEBUG (AutoscalingActionExecutor-450-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_restore_trigger after 100ms
[junit4] 2> 449450 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 1
[junit4] 2> 449450 DEBUG (AutoscalingActionExecutor-450-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 116 ms for event id=54555a9a920d9T9ytwjybgp40jbpq7iexv6qgmb
[junit4] 2> 449453 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testSearchRate
[junit4] 2> 449453 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 449454 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 449454 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10004_solr
[junit4] 2> 449454 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 449454 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 449454 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 449454 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:163)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 449454 DEBUG (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 449500 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .autoAddReplicas trigger
[junit4] 2> 449500 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 449500 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 449500 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion -1
[junit4] 2> 449500 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 449501 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 449501 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 10, lastZnodeVersion 10
[junit4] 2> 449501 DEBUG (ScheduledTrigger-452-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449521 DEBUG (ScheduledTrigger-452-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449542 DEBUG (ScheduledTrigger-452-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449562 DEBUG (ScheduledTrigger-452-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449582 DEBUG (ScheduledTrigger-452-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449602 DEBUG (ScheduledTrigger-452-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449623 DEBUG (ScheduledTrigger-452-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449643 DEBUG (ScheduledTrigger-452-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449643 DEBUG (simCloudManagerPool-451-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 449644 DEBUG (simCloudManagerPool-451-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 449647 DEBUG (simCloudManagerPool-451-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 449647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 449647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 449647 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 449648 DEBUG (simCloudManagerPool-451-thread-5) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 449648 DEBUG (simCloudManagerPool-451-thread-5) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 449649 DEBUG (simCloudManagerPool-451-thread-5) [ ] o.a.s.c.a.NodeLostTrigger Initial livenodes: [127.0.0.1:10001_solr, 127.0.0.1:10002_solr]
[junit4] 2> 449649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 449649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 449649 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 449663 DEBUG (ScheduledTrigger-452-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449683 DEBUG (ScheduledTrigger-452-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449704 DEBUG (ScheduledTrigger-452-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449724 DEBUG (ScheduledTrigger-452-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 449743 DEBUG (ScheduledTrigger-452-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"54558c1a16063T9ytwjybgp40jbpq7iexv6qgme",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1483622391570531,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10001_solr":250.0,
[junit4] 2> "127.0.0.1:10002_solr":250.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10001_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10002_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}}
[junit4] 2> 449744 DEBUG (ScheduledTrigger-452-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, search_rate_trigger]
[junit4] 2> 449745 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"54558c1a16063T9ytwjybgp40jbpq7iexv6qgme",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1483622391570531,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10001_solr":250.0,
[junit4] 2> "127.0.0.1:10002_solr":250.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10001_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10002_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "_enqueue_time_":1483626976635481,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}}
[junit4] 2> 449745 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"54558c1a16063T9ytwjybgp40jbpq7iexv6qgme",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1483622391570531,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10001_solr":250.0,
[junit4] 2> "127.0.0.1:10002_solr":250.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10001_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10002_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "_enqueue_time_":1483626976635481,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}} with context properties: {BEFORE_ACTION=[compute]}
[junit4] 2> 449767 INFO (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10002_solr&type=NRT
[junit4] 2> 449780 INFO (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10001_solr&type=NRT
[junit4] 2> 449780 INFO (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10001_solr&type=NRT
[junit4] 2> 449780 INFO (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 1483628797 sessionWrapper.createTime 1483627245750481, this.sessionWrapper.createTime 1483627245750481
[junit4] 2> 449781 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"54558c1a16063T9ytwjybgp40jbpq7iexv6qgme",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1483622391570531,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10001_solr":250.0,
[junit4] 2> "127.0.0.1:10002_solr":250.0},
[junit4] 2> "__start__":1,
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10001_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10002_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "_enqueue_time_":1483626976635481,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@613fd549, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@f20fae2, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@417612c7], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
[junit4] 2> 449781 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10002_solr&type=NRT
[junit4] 2> 449807 INFO (simCloudManagerPool-451-thread-7) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10002_solr for creating new replica
[junit4] 2> 449837 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10001_solr&type=NRT
[junit4] 2> 449857 INFO (simCloudManagerPool-451-thread-11) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10001_solr for creating new replica
[junit4] 2> 449858 DEBUG (simCloudManagerPool-451-thread-8) [ ] o.a.s.c.a.s.SimClusterStateProvider -- elected new leader for collection1 / shard1: {"core_node1":{
[junit4] 2> "core":"collection1_shard1_replica_n1",
[junit4] 2> "leader":"true",
[junit4] 2> "INDEX.sizeInBytes":1000000000,
[junit4] 2> "rate":250.0,
[junit4] 2> "node_name":"127.0.0.1:10002_solr",
[junit4] 2> "state":"active",
[junit4] 2> "type":"NRT",
[junit4] 2> "QUERY./select.requestTimes:1minRate":250.0,
[junit4] 2> "shard":"shard1",
[junit4] 2> "collection":"collection1"}}
[junit4] 2> 449894 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10001_solr&type=NRT
[junit4] 2> 449918 INFO (simCloudManagerPool-451-thread-15) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10001_solr for creating new replica
[junit4] 2> 449931 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 449931 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: search_rate_trigger after 100ms
[junit4] 2> 449931 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 9304 ms for event id=54558c1a16063T9ytwjybgp40jbpq7iexv6qgme
[junit4] 2> 450031 DEBUG (ScheduledTrigger-452-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 450032 DEBUG (ScheduledTrigger-452-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"5455997a3cd95T9ytwjybgp40jbpq7iexv6qgmh",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1483625982053781,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10001_solr":250.0,
[junit4] 2> "127.0.0.1:10002_solr":250.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10001_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10002_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}}
[junit4] 2> 450032 DEBUG (ScheduledTrigger-452-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, search_rate_trigger]
[junit4] 2> 450034 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testSearchRate
[junit4] 2> 450034 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase
[junit4] 2>
[junit4] 2> 450034 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2> 450034 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ############ FINAL CLUSTER STATS ############
[junit4] 2> 450034 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #############################################
[junit4] 2>
[junit4] 2> 450034 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Live nodes: 2
[junit4] 2> 450034 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"5455997a3cd95T9ytwjybgp40jbpq7iexv6qgmh",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1483625982053781,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10001_solr":250.0,
[junit4] 2> "127.0.0.1:10002_solr":250.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10001_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10002_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "_enqueue_time_":1483641407873081,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}}
[junit4] 2> 450034 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ComputePlanAction -- processing event: {
[junit4] 2> "id":"5455997a3cd95T9ytwjybgp40jbpq7iexv6qgmh",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1483625982053781,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10001_solr":250.0,
[junit4] 2> "127.0.0.1:10002_solr":250.0},
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10001_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10002_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "_enqueue_time_":1483641407873081,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}} with context properties: {BEFORE_ACTION=[compute]}
[junit4] 2> 450034 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Empty nodes: 0
[junit4] 2> 450035 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Dead nodes: 0
[junit4] 2> 450035 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Collections: [collection1]
[junit4] 2> 450035 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Max replicas per node: 3
[junit4] 2> 450035 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Min replicas per node: 2
[junit4] 2> 450035 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## Total replicas: 5
[junit4] 2> 450036 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * collection1 5
[junit4] 2> 450036 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - active 5
[junit4] 2> 450036 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Final Solr op counts ##########
[junit4] 2> 450036 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - ADDREPLICA 3
[junit4] 2> 450036 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - CREATE 1
[junit4] 2> 450036 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - DELETESTATUS 3
[junit4] 2> 450036 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - REQUESTSTATUS 3
[junit4] 2> 450036 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - autoscaling 2
[junit4] 2> 450036 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - update 3
[junit4] 2> 450037 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ######### Autoscaling event counts ###########
[junit4] 2> 450037 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## * Trigger: search_rate_trigger
[junit4] 2> 450037 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - STARTED 2
[junit4] 2> 450037 INFO (TEST-TestTriggerIntegration.testSearchRate-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase ## - SUCCEEDED 1
[junit4] 2> 450035 INFO (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10002_solr&type=NRT
[junit4] 2> 450038 INFO (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10002_solr&type=NRT
[junit4] 2> 450038 INFO (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ComputePlanAction Computed Plan: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10002_solr&type=NRT
[junit4] 2> 450038 INFO (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.s.c.a.PolicyHelper returnSession, curr-time 1483641686 sessionWrapper.createTime 1483641547419931, this.sessionWrapper.createTime 1483641547419931
[junit4] 2> 450038 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ExecutePlanAction -- processing event: {
[junit4] 2> "id":"5455997a3cd95T9ytwjybgp40jbpq7iexv6qgmh",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1483625982053781,
[junit4] 2> "eventType":"SEARCHRATE",
[junit4] 2> "properties":{
[junit4] 2> "node":{
[junit4] 2> "127.0.0.1:10001_solr":250.0,
[junit4] 2> "127.0.0.1:10002_solr":250.0},
[junit4] 2> "__start__":1,
[junit4] 2> "replica":[
[junit4] 2> "{\"core_node2\":{\n \"core\":\"collection1_shard1_replica_n2\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10001_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}",
[junit4] 2> "{\"core_node1\":{\n \"core\":\"collection1_shard1_replica_n1\",\n \"leader\":\"true\",\n \"INDEX.sizeInBytes\":1000000000,\n \"rate\":250.0,\n \"node_name\":\"127.0.0.1:10002_solr\",\n \"state\":\"active\",\n \"type\":\"NRT\",\n \"QUERY./select.requestTimes:1minRate\":250.0,\n \"shard\":\"shard1\",\n \"collection\":\"collection1\"}}"],
[junit4] 2> "collection":{"collection1":500.0},
[junit4] 2> "shard":{"collection1":{"shard1":500.0}},
[junit4] 2> "_enqueue_time_":1483641407873081,
[junit4] 2> "requestedOps":[
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}",
[junit4] 2> "Op{action=ADDREPLICA, hints={COLL_SHARD={\n \"first\":\"collection1\",\n \"second\":\"shard1\"}}}"]}} with context properties: {operations=[org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@d7cdefa, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@fb897aa, org.apache.solr.client.solrj.request.CollectionAdminRequest$AddReplica@16344673], AFTER_ACTION=[compute], BEFORE_ACTION=[compute, execute]}
[junit4] 2> 450038 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10002_solr&type=NRT
[junit4] 2> 450073 INFO (simCloudManagerPool-451-thread-21) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10002_solr for creating new replica
[junit4] 2> 450087 DEBUG (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ExecutePlanAction Executing operation: action=ADDREPLICA&collection=collection1&shard=shard1&node=127.0.0.1:10002_solr&type=NRT
[junit4] 2> 450087 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTrigger
[junit4] 2> 450087 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 450087 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 450088 INFO (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 450088 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 450088 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 450088 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220)
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335)
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:163)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 450088 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTrigger-seed#[A83ABF3BE9CAEDB3]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 450088 INFO (simCloudManagerPool-451-thread-25) [ ] o.a.s.c.a.c.AddReplicaCmd Node Identified 127.0.0.1:10002_solr for creating new replica
[junit4] 2> 450105 WARN (AutoscalingActionExecutor-453-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> java.lang.Exception: Error executing action: execute for trigger event: {
[junit4] 2> "id":"5455997a3cd95T9ytwjybgp40jbpq7iexv6qgmh",
[junit4] 2> "source":"search_rate_trigger",
[junit4] 2> "eventTime":1483625982053781,
[...truncated too long message...]
new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-dataimporthandler/test/J2/temp/solr.handler.dataimport.TestContentStreamDataSource_7C0C821FF129BF1A-001/inst-002/collection1], dataDir=[/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-dataimporthandler/test/J2/temp/solr.handler.dataimport.TestContentStreamDataSource_7C0C821FF129BF1A-001/inst-002/collection1/data/]
[junit4] 2> 90318 T88 x:x:collection1 oasu.CommitTracker.<init> Hard AutoCommit: disabled
[junit4] 2> 90318 T88 x:x:collection1 oasu.CommitTracker.<init> Soft AutoCommit: disabled
[junit4] 2> 90320 T88 x:x:collection1 oass.SolrIndexSearcher.<init> Opening [Searcher@6044c6cf[collection1] main]
[junit4] 2> 90320 T88 x:x:collection1 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-dataimporthandler/test/J2/temp/solr.handler.dataimport.TestContentStreamDataSource_7C0C821FF129BF1A-001/inst-002/collection1/conf
[junit4] 2> 90339 T88 x:x:collection1 oash.ReplicationHandler.inform Commits will be reserved for 10000ms.
[junit4] 2> 90502 T92 x:x:collection1 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6044c6cf[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> ASYNC NEW_CORE C6 name=collection1 org.apache.solr.core.SolrCore@7f29e215
[junit4] 2> 91132 T81 x:x:collection1 C6 oashd.DataImporter.maybeReloadConfiguration Loading DIH Configuration: data-config.xml
[junit4] 2> 91134 T81 x:x:collection1 C6 oashd.DataImporter.loadDataConfig Data Configuration loaded successfully
[junit4] 2> 91134 T81 x:x:collection1 C6 oashd.DataImporter.doFullImport Starting Full Import
[junit4] 2> 91135 T81 x:x:collection1 C6 oashd.SimplePropertiesWriter.readIndexerProperties WARN Unable to read: dataimport.properties
[junit4] 2> 91264 T81 x:x:collection1 C6 oashd.DocBuilder.finish Import completed successfully
[junit4] 2> 91264 T81 x:x:collection1 C6 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 91264 T81 x:x:collection1 C6 oasu.SolrIndexWriter.setCommitData Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@265f4519 commitCommandVersion:0
[junit4] 2> 91301 T81 x:x:collection1 C6 oass.SolrIndexSearcher.<init> Opening [Searcher@70cb2f87[collection1] main]
[junit4] 2> 91302 T81 x:x:collection1 C6 oasu.DirectUpdateHandler2.commit end_commit_flush
[junit4] 2> 91322 T92 x:x:collection1 C6 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@70cb2f87[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):c2)))}
[junit4] 2> 91323 T81 x:x:collection1 C6 oashd.SimplePropertiesWriter.readIndexerProperties WARN Unable to read: dataimport.properties
[junit4] 2> 91356 T81 x:x:collection1 C6 oashd.SimplePropertiesWriter.persist Wrote last indexed time to dataimport.properties
[junit4] 2> 91356 T81 x:x:collection1 C6 oashd.DocBuilder.execute Time taken = 0:0:0.221
[junit4] 2> 91357 T81 x:x:collection1 C6 oasup.LogUpdateProcessorFactory$LogUpdateProcessor.finish [collection1] webapp=/solr path=/dataimport params={clean=false&wt=javabin&version=2&command=full-import}{add=[1, 2],commit=} 0 225
[junit4] 2> 91403 T83 x:x:collection1 C6 oasc.SolrCore.execute [collection1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=2 status=0 QTime=0
[junit4] 2> 91446 T54 oejs.AbstractConnector.doStop Stopped ServerConnector@63d9ad41{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 91446 T54 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=28799930
[junit4] 2> 91465 T54 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 91466 T54 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@38d58723: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@56856d67
[junit4] 2> 91503 T54 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 91503 T54 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@44bedb71: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1eb78b2d
[junit4] 2> 91524 T54 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 91525 T54 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@771553e8: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@599827a3
[junit4] 2> 91612 T96 x:x:collection1 oasc.SolrCore.close [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@7f29e215
[junit4] 2> 91612 T96 x:x:collection1 oasm.SolrMetricManager.closeReporters Closing metric reporters for registry=solr.core.collection1, tag=2133451285
[junit4] 2> 91612 T96 x:x:collection1 oasmr.SolrJmxReporter.close Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4e9c5bc8: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@49682f91
[junit4] 2> 91750 T54 oejsh.ContextHandler.doStop Stopped o.e.j.s.ServletContextHandler@3f6abfe{/solr,null,UNAVAILABLE}
[junit4] 2> 91751 T54 oejss.HouseKeeper.stopScavenging Stopped scavenging
[junit4] 2> 91828 T54 oas.SolrTestCaseJ4.tearDown ###Ending testSimple
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-dataimporthandler/test/J2/temp/solr.handler.dataimport.TestContentStreamDataSource_7C0C821FF129BF1A-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@2ccaf83e), locale=tr, timezone=Asia/Ulaanbaatar
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=1,free=207408344,total=366477312
[junit4] 2> NOTE: All tests run in this JVM: [TestPlainTextEntityProcessor, TestDateFormatTransformer, TestEphemeralCache, TestClobTransformer, TestVariableResolver, TestURLDataSource, TestXPathEntityProcessor, TestRegexTransformer, TestSqlEntityProcessorDelta, TestContentStreamDataSource]
[junit4] Completed [23/38 (1!)] on J2 in 16.35s, 2 tests, 1 failure <<< FAILURES!
[...truncated 2347 lines...]
[javac] Compiling 6 source files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build/contrib/solr-prometheus-exporter/classes/java
[javac] /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/prometheus-exporter/src/java/org/apache/solr/prometheus/collector/SolrCollector.java:56: error: cannot find symbol
[javac] private static final Logger logger = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
[javac] ^
[javac] symbol: variable MethodHandles
[javac] location: class SolrCollector
[javac] /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/prometheus-exporter/src/java/org/apache/solr/prometheus/exporter/SolrExporter.java:52: error: cannot find symbol
[javac] private static final Logger logger = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
[javac] ^
[javac] symbol: variable MethodHandles
[javac] location: class SolrExporter
[javac] /home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/contrib/prometheus-exporter/src/java/org/apache/solr/prometheus/scraper/SolrScraper.java:49: error: cannot find symbol
[javac] private static final Logger logger = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass());
[javac] ^
[javac] symbol: variable MethodHandles
[javac] location: class SolrScraper
[javac] 3 errors
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:618: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:562: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/build.xml:59: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/build.xml:282: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/common-build.xml:478: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/solr/common-build.xml:557: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:544: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-Tests-master/lucene/common-build.xml:2044: Compile failed; see the compiler error output for details.
Total time: 196 minutes 13 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any