You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/04/12 16:12:50 UTC
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 39 - Still
Failing
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/39/
3 tests failed.
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
Test abandoned because suite timeout was reached.
Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
at __randomizedtesting.SeedInfo.seed([D93B52828EC50BB8]:0)
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.HttpPartitionTest
Error Message:
Suite timeout exceeded (>= 7200000 msec).
Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
at __randomizedtesting.SeedInfo.seed([D93B52828EC50BB8]:0)
FAILED: org.apache.solr.cloud.MoveReplicaHDFSTest.testFailedMove
Error Message:
No live SolrServers available to handle this request:[http://127.0.0.1:43008/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:37177/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:42851/solr/MoveReplicaHDFSTest_failed_coll_true]
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: No live SolrServers available to handle this request:[http://127.0.0.1:43008/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:37177/solr/MoveReplicaHDFSTest_failed_coll_true, http://127.0.0.1:42851/solr/MoveReplicaHDFSTest_failed_coll_true]
at __randomizedtesting.SeedInfo.seed([D93B52828EC50BB8:73F681703916DE68]:0)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:462)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:993)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrClient.query(SolrClient.java:942)
at org.apache.solr.cloud.MoveReplicaTest.testFailedMove(MoveReplicaTest.java:308)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42851/solr/MoveReplicaHDFSTest_failed_coll_true: Error from server at http://127.0.0.1:37177/solr/MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2/select</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2/select. Reason:
<pre> Can not find: /solr/MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2/select</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.8.v20171121</a><hr/>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
... 46 more
Build Log:
[...truncated 13766 lines...]
[junit4] Suite: org.apache.solr.cloud.MoveReplicaHDFSTest
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/init-core-data-001
[junit4] 2> 2846977 WARN (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=6 numCloses=6
[junit4] 2> 2846977 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2846978 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2846978 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2846979 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 4 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-001
[junit4] 2> 2846979 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2846979 INFO (Thread-4640) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2846979 INFO (Thread-4640) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2846983 ERROR (Thread-4640) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2847079 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:42248
[junit4] 2> 2847084 INFO (zkConnectionManagerCallback-7323-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847091 INFO (jetty-launcher-7320-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2847091 INFO (jetty-launcher-7320-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2847091 INFO (jetty-launcher-7320-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2847091 INFO (jetty-launcher-7320-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2847092 INFO (jetty-launcher-7320-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2847092 INFO (jetty-launcher-7320-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2847092 INFO (jetty-launcher-7320-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 2847093 INFO (jetty-launcher-7320-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2847093 INFO (jetty-launcher-7320-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2847093 INFO (jetty-launcher-7320-thread-3) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2847093 INFO (jetty-launcher-7320-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2da18c21{/solr,null,AVAILABLE}
[junit4] 2> 2847093 INFO (jetty-launcher-7320-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2847093 INFO (jetty-launcher-7320-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2847093 INFO (jetty-launcher-7320-thread-1) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2847093 INFO (jetty-launcher-7320-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2847093 INFO (jetty-launcher-7320-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2847093 INFO (jetty-launcher-7320-thread-4) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2847093 INFO (jetty-launcher-7320-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2eb37e71{/solr,null,AVAILABLE}
[junit4] 2> 2847094 INFO (jetty-launcher-7320-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@4d09c515{HTTP/1.1,[http/1.1]}{127.0.0.1:39928}
[junit4] 2> 2847094 INFO (jetty-launcher-7320-thread-2) [ ] o.e.j.s.Server Started @2847210ms
[junit4] 2> 2847094 INFO (jetty-launcher-7320-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39928}
[junit4] 2> 2847094 ERROR (jetty-launcher-7320-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2847094 INFO (jetty-launcher-7320-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2847094 INFO (jetty-launcher-7320-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 2847094 INFO (jetty-launcher-7320-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2847094 INFO (jetty-launcher-7320-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2847094 INFO (jetty-launcher-7320-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-12T14:37:17.495Z
[junit4] 2> 2847095 INFO (jetty-launcher-7320-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@547c78a7{/solr,null,AVAILABLE}
[junit4] 2> 2847095 INFO (jetty-launcher-7320-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@104449f3{/solr,null,AVAILABLE}
[junit4] 2> 2847095 INFO (jetty-launcher-7320-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5942d19{HTTP/1.1,[http/1.1]}{127.0.0.1:37177}
[junit4] 2> 2847096 INFO (jetty-launcher-7320-thread-3) [ ] o.e.j.s.Server Started @2847211ms
[junit4] 2> 2847096 INFO (jetty-launcher-7320-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37177}
[junit4] 2> 2847096 ERROR (jetty-launcher-7320-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2847096 INFO (jetty-launcher-7320-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2847096 INFO (jetty-launcher-7320-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 2847096 INFO (jetty-launcher-7320-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2847096 INFO (jetty-launcher-7320-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2847096 INFO (jetty-launcher-7320-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-12T14:37:17.497Z
[junit4] 2> 2847097 INFO (jetty-launcher-7320-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@41108655{HTTP/1.1,[http/1.1]}{127.0.0.1:43887}
[junit4] 2> 2847097 INFO (jetty-launcher-7320-thread-1) [ ] o.e.j.s.Server Started @2847212ms
[junit4] 2> 2847097 INFO (jetty-launcher-7320-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43887}
[junit4] 2> 2847097 ERROR (jetty-launcher-7320-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2847097 INFO (jetty-launcher-7320-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2847097 INFO (jetty-launcher-7320-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 2847097 INFO (jetty-launcher-7320-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2847097 INFO (jetty-launcher-7320-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2847097 INFO (jetty-launcher-7320-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-12T14:37:17.498Z
[junit4] 2> 2847098 INFO (jetty-launcher-7320-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2d587425{HTTP/1.1,[http/1.1]}{127.0.0.1:42851}
[junit4] 2> 2847098 INFO (jetty-launcher-7320-thread-4) [ ] o.e.j.s.Server Started @2847213ms
[junit4] 2> 2847098 INFO (jetty-launcher-7320-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=42851}
[junit4] 2> 2847098 ERROR (jetty-launcher-7320-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2847098 INFO (jetty-launcher-7320-thread-4) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2847098 INFO (zkConnectionManagerCallback-7325-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847098 INFO (jetty-launcher-7320-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 2847098 INFO (jetty-launcher-7320-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2847098 INFO (jetty-launcher-7320-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2847098 INFO (jetty-launcher-7320-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-12T14:37:17.499Z
[junit4] 2> 2847098 INFO (zkConnectionManagerCallback-7329-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847099 INFO (jetty-launcher-7320-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2847141 INFO (zkConnectionManagerCallback-7327-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847141 INFO (zkConnectionManagerCallback-7331-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847142 INFO (jetty-launcher-7320-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2847142 INFO (jetty-launcher-7320-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2847142 INFO (jetty-launcher-7320-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2847156 INFO (jetty-launcher-7320-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42248/solr
[junit4] 2> 2847159 INFO (jetty-launcher-7320-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42248/solr
[junit4] 2> 2847159 INFO (jetty-launcher-7320-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42248/solr
[junit4] 2> 2847162 INFO (zkConnectionManagerCallback-7335-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847172 INFO (zkConnectionManagerCallback-7343-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847172 INFO (zkConnectionManagerCallback-7341-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847172 INFO (jetty-launcher-7320-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42248/solr
[junit4] 2> 2847176 INFO (zkConnectionManagerCallback-7346-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847176 INFO (zkConnectionManagerCallback-7349-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847177 INFO (zkConnectionManagerCallback-7351-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847178 INFO (zkConnectionManagerCallback-7353-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847179 INFO (zkConnectionManagerCallback-7355-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847230 INFO (jetty-launcher-7320-thread-1) [n:127.0.0.1:43887_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2847231 INFO (jetty-launcher-7320-thread-1) [n:127.0.0.1:43887_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43887_solr
[junit4] 2> 2847232 INFO (jetty-launcher-7320-thread-1) [n:127.0.0.1:43887_solr ] o.a.s.c.Overseer Overseer (id=72443026872336396-127.0.0.1:43887_solr-n_0000000000) starting
[junit4] 2> 2847262 INFO (zkConnectionManagerCallback-7368-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847269 INFO (jetty-launcher-7320-thread-1) [n:127.0.0.1:43887_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42248/solr ready
[junit4] 2> 2847270 INFO (jetty-launcher-7320-thread-1) [n:127.0.0.1:43887_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43887_solr
[junit4] 2> 2847306 INFO (OverseerStateUpdate-72443026872336396-127.0.0.1:43887_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2847352 INFO (zkCallback-7367-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2847394 INFO (jetty-launcher-7320-thread-4) [n:127.0.0.1:42851_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2847396 INFO (jetty-launcher-7320-thread-3) [n:127.0.0.1:37177_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2847397 INFO (jetty-launcher-7320-thread-4) [n:127.0.0.1:42851_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2847399 INFO (jetty-launcher-7320-thread-3) [n:127.0.0.1:37177_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2847408 INFO (jetty-launcher-7320-thread-4) [n:127.0.0.1:42851_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2847408 INFO (jetty-launcher-7320-thread-4) [n:127.0.0.1:42851_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42851_solr
[junit4] 2> 2847409 INFO (zkCallback-7367-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2847410 INFO (zkCallback-7354-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2847418 INFO (jetty-launcher-7320-thread-2) [n:127.0.0.1:39928_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2847419 INFO (jetty-launcher-7320-thread-2) [n:127.0.0.1:39928_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2847421 INFO (jetty-launcher-7320-thread-2) [n:127.0.0.1:39928_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2847421 INFO (jetty-launcher-7320-thread-2) [n:127.0.0.1:39928_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39928_solr
[junit4] 2> 2847435 INFO (zkCallback-7367-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2847448 INFO (zkCallback-7352-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 2847450 INFO (zkCallback-7354-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2847466 INFO (zkCallback-7345-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2847485 INFO (jetty-launcher-7320-thread-3) [n:127.0.0.1:37177_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2847485 INFO (jetty-launcher-7320-thread-3) [n:127.0.0.1:37177_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37177_solr
[junit4] 2> 2847487 INFO (zkCallback-7345-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2847487 INFO (zkCallback-7354-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2847519 INFO (zkCallback-7350-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 2847549 INFO (zkCallback-7367-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2847554 INFO (zkCallback-7352-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2847554 INFO (zkCallback-7350-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2847767 INFO (jetty-launcher-7320-thread-1) [n:127.0.0.1:43887_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43887.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2847773 INFO (jetty-launcher-7320-thread-2) [n:127.0.0.1:39928_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39928.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2847780 INFO (jetty-launcher-7320-thread-2) [n:127.0.0.1:39928_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39928.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2847780 INFO (jetty-launcher-7320-thread-2) [n:127.0.0.1:39928_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39928.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2847781 INFO (jetty-launcher-7320-thread-2) [n:127.0.0.1:39928_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-001/node2/.
[junit4] 2> 2847783 INFO (jetty-launcher-7320-thread-1) [n:127.0.0.1:43887_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43887.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2847783 INFO (jetty-launcher-7320-thread-1) [n:127.0.0.1:43887_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43887.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2847784 INFO (jetty-launcher-7320-thread-1) [n:127.0.0.1:43887_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-001/node1/.
[junit4] 2> 2847792 INFO (zkConnectionManagerCallback-7373-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847794 INFO (jetty-launcher-7320-thread-2) [n:127.0.0.1:39928_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2847795 INFO (jetty-launcher-7320-thread-2) [n:127.0.0.1:39928_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42248/solr ready
[junit4] 2> 2847858 INFO (jetty-launcher-7320-thread-4) [n:127.0.0.1:42851_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42851.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2847865 INFO (jetty-launcher-7320-thread-4) [n:127.0.0.1:42851_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42851.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2847865 INFO (jetty-launcher-7320-thread-4) [n:127.0.0.1:42851_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42851.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2847866 INFO (jetty-launcher-7320-thread-4) [n:127.0.0.1:42851_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-001/node4/.
[junit4] 2> 2847869 INFO (zkConnectionManagerCallback-7378-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2847870 INFO (jetty-launcher-7320-thread-4) [n:127.0.0.1:42851_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2847870 INFO (jetty-launcher-7320-thread-4) [n:127.0.0.1:42851_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42248/solr ready
[junit4] 2> 2848004 INFO (jetty-launcher-7320-thread-3) [n:127.0.0.1:37177_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37177.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2848011 INFO (jetty-launcher-7320-thread-3) [n:127.0.0.1:37177_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37177.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2848011 INFO (jetty-launcher-7320-thread-3) [n:127.0.0.1:37177_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37177.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2848012 INFO (jetty-launcher-7320-thread-3) [n:127.0.0.1:37177_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-001/node3/.
[junit4] 2> 2848015 INFO (zkConnectionManagerCallback-7383-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2848016 INFO (jetty-launcher-7320-thread-3) [n:127.0.0.1:37177_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2848017 INFO (jetty-launcher-7320-thread-3) [n:127.0.0.1:37177_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42248/solr ready
[junit4] 2> 2848064 INFO (zkConnectionManagerCallback-7385-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2848067 INFO (zkConnectionManagerCallback-7390-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2848068 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2848068 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42248/solr ready
[junit4] 2> 2848072 INFO (qtp2003698812-22787) [n:127.0.0.1:43887_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :overseerstatus with params action=OVERSEERSTATUS&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2848100 INFO (qtp2003698812-22787) [n:127.0.0.1:43887_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={action=OVERSEERSTATUS&wt=javabin&version=2} status=0 QTime=28
[junit4] 1> Formatting using clusterid: testClusterID
[junit4] 2> 2848181 WARN (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.h.m.i.MetricsConfig Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties
[junit4] 2> 2848207 WARN (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 2848208 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 2848240 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/hdfs to ./temp/Jetty_lucene2.us.west_apache_org_34931_hdfs____5whc5c/webapp
[junit4] 2> 2848730 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@lucene2-us-west.apache.org:34931
[junit4] 2> 2849433 WARN (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 2849434 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 2849542 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_33839_datanode____p3mbs5/webapp
[junit4] 2> 2850114 INFO (OverseerCollectionConfigSetProcessor-72443026872336396-127.0.0.1:43887_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2850595 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:33839
[junit4] 2> 2851459 WARN (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.a.h.h.HttpRequestLog Jetty request log can only be enabled using Log4j
[junit4] 2> 2851464 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.m.log jetty-6.1.26
[junit4] 2> 2851602 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.m.log Extract jar:file:/home/jenkins/.ivy2/cache/org.apache.hadoop/hadoop-hdfs/tests/hadoop-hdfs-2.7.4-tests.jar!/webapps/datanode to ./temp/Jetty_localhost_35699_datanode____.ksa1np/webapp
[junit4] 2> 2852079 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-002/hdfsBaseDir/data/data1/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-002/hdfsBaseDir/data/data2/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:46819) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 2852185 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x14e4f5b6edf74f: from storage DS-eb984553-996c-4b9b-b087-af607130edec node DatanodeRegistration(127.0.0.1:34546, datanodeUuid=6b7471b1-d0bc-4eea-b1b9-b210f1e68c0a, infoPort=37266, infoSecurePort=0, ipcPort=41411, storageInfo=lv=-56;cid=testClusterID;nsid=1825728911;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 2852185 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x14e4f5b6edf74f: from storage DS-4bf5ee87-9bf2-4247-81f0-6922a233962a node DatanodeRegistration(127.0.0.1:34546, datanodeUuid=6b7471b1-d0bc-4eea-b1b9-b210f1e68c0a, infoPort=37266, infoSecurePort=0, ipcPort=41411, storageInfo=lv=-56;cid=testClusterID;nsid=1825728911;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 2852546 INFO (SUITE-MoveReplicaHDFSTest-seed#[D93B52828EC50BB8]-worker) [ ] o.m.log Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:35699
[junit4] 2> 2852763 ERROR (DataNode: [[[DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-002/hdfsBaseDir/data/data3/, [DISK]file:/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-002/hdfsBaseDir/data/data4/]] heartbeating to lucene2-us-west.apache.org/127.0.0.1:46819) [ ] o.a.h.h.s.d.DirectoryScanner dfs.datanode.directoryscan.throttle.limit.ms.per.sec set to value below 1 ms/sec. Assuming default value of 1000
[junit4] 2> 2852792 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x14e4f5db36aa1f: from storage DS-9dafaf1f-cce1-4816-acff-7f3923c3c568 node DatanodeRegistration(127.0.0.1:40655, datanodeUuid=59ea0808-3440-4e98-abda-4cbb700557dc, infoPort=37348, infoSecurePort=0, ipcPort=41828, storageInfo=lv=-56;cid=testClusterID;nsid=1825728911;c=0), blocks: 0, hasStaleStorage: true, processing time: 0 msecs
[junit4] 2> 2852792 INFO (Block report processor) [ ] BlockStateChange BLOCK* processReport 0x14e4f5db36aa1f: from storage DS-14749d85-1b31-40d5-a4e6-eba358d92531 node DatanodeRegistration(127.0.0.1:40655, datanodeUuid=59ea0808-3440-4e98-abda-4cbb700557dc, infoPort=37348, infoSecurePort=0, ipcPort=41828, storageInfo=lv=-56;cid=testClusterID;nsid=1825728911;c=0), blocks: 0, hasStaleStorage: false, processing time: 0 msecs
[junit4] 2> 2853174 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testFailedMove
[junit4] 2> 2853175 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2853176 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T16:27:37-05:00, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 2853177 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 2853177 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 2853177 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 2853177 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@c2f02ff{/solr,null,AVAILABLE}
[junit4] 2> 2853178 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@11324d3c{HTTP/1.1,[http/1.1]}{127.0.0.1:43008}
[junit4] 2> 2853178 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.e.j.s.Server Started @2853294ms
[junit4] 2> 2853178 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43008}
[junit4] 2> 2853178 ERROR (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2853178 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 2853178 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 2853178 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2853178 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2853178 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-12T14:37:23.579Z
[junit4] 2> 2853180 INFO (zkConnectionManagerCallback-7394-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2853180 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2853200 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:42248/solr
[junit4] 2> 2853201 INFO (zkConnectionManagerCallback-7398-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2853203 INFO (zkConnectionManagerCallback-7400-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2853208 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [n:127.0.0.1:43008_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 2853210 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [n:127.0.0.1:43008_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2853211 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [n:127.0.0.1:43008_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2853211 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [n:127.0.0.1:43008_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43008_solr
[junit4] 2> 2853213 INFO (zkCallback-7352-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2853213 INFO (zkCallback-7354-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2853213 INFO (zkCallback-7345-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2853213 INFO (zkCallback-7367-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2853248 INFO (zkCallback-7377-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2853248 INFO (zkCallback-7350-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2853248 INFO (zkCallback-7399-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2853248 INFO (zkCallback-7372-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2853256 INFO (zkCallback-7389-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2853257 INFO (zkCallback-7389-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2853257 INFO (zkCallback-7382-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 2853516 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [n:127.0.0.1:43008_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43008.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2853525 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [n:127.0.0.1:43008_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43008.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2853525 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [n:127.0.0.1:43008_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43008.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2853527 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [n:127.0.0.1:43008_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-001/node5/.
[junit4] 2> 2853537 INFO (zkConnectionManagerCallback-7407-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2853538 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [n:127.0.0.1:43008_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 2853539 INFO (TEST-MoveReplicaHDFSTest.testFailedMove-seed#[D93B52828EC50BB8]) [n:127.0.0.1:43008_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:42248/solr ready
[junit4] 2> 2855174 INFO (zkConnectionManagerCallback-7409-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2855177 INFO (qtp1603064068-22788) [n:127.0.0.1:37177_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf1&autoAddReplicas=false&name=MoveReplicaHDFSTest_failed_coll_true&nrtReplicas=2&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2855179 INFO (OverseerThreadFactory-8209-thread-2) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection MoveReplicaHDFSTest_failed_coll_true
[junit4] 2> 2855290 INFO (OverseerStateUpdate-72443026872336396-127.0.0.1:43887_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43887/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2855292 INFO (OverseerStateUpdate-72443026872336396-127.0.0.1:43887_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:37177/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2855294 INFO (OverseerStateUpdate-72443026872336396-127.0.0.1:43887_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:42851/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2855309 INFO (OverseerStateUpdate-72443026872336396-127.0.0.1:43887_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MoveReplicaHDFSTest_failed_coll_true",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43008/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2855511 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2855511 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2855512 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2855515 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2855515 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf1&newCollection=true&name=MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6&action=CREATE&numShards=2&collection=MoveReplicaHDFSTest_failed_coll_true&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2855640 INFO (zkCallback-7399-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2855640 INFO (zkCallback-7354-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2855640 INFO (zkCallback-7350-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2855640 INFO (zkCallback-7354-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2855640 INFO (zkCallback-7352-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2855640 INFO (zkCallback-7350-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2855640 INFO (zkCallback-7399-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2855641 INFO (zkCallback-7352-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2856547 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2856547 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2856570 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2856570 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.4.0
[junit4] 2> 2856575 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] Schema name=minimal
[junit4] 2> 2856577 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] Schema name=minimal
[junit4] 2> 2856589 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2856589 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6' using configuration from collection MoveReplicaHDFSTest_failed_coll_true, trusted=true
[junit4] 2> 2856590 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43008.solr.core.MoveReplicaHDFSTest_failed_coll_true.shard2.replica_n6' (registry 'solr.core.MoveReplicaHDFSTest_failed_coll_true.shard2.replica_n6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2856590 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:46819/data
[junit4] 2> 2856590 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 2856590 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2856590 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-001/node5/MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6], dataDir=[hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node8/data/]
[junit4] 2> 2856592 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node8/data/snapshot_metadata
[junit4] 2> 2856599 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2856599 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4' using configuration from collection MoveReplicaHDFSTest_failed_coll_true, trusted=true
[junit4] 2> 2856599 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_42851.solr.core.MoveReplicaHDFSTest_failed_coll_true.shard2.replica_n4' (registry 'solr.core.MoveReplicaHDFSTest_failed_coll_true.shard2.replica_n4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2856600 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] Schema name=minimal
[junit4] 2> 2856600 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:46819/data
[junit4] 2> 2856600 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 2856600 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2856600 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-001/node4/MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4], dataDir=[hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node7/data/]
[junit4] 2> 2856601 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node7/data/snapshot_metadata
[junit4] 2> 2856601 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2856601 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2' using configuration from collection MoveReplicaHDFSTest_failed_coll_true, trusted=true
[junit4] 2> 2856615 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.s.IndexSchema [MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2856665 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37177.solr.core.MoveReplicaHDFSTest_failed_coll_true.shard1.replica_n2' (registry 'solr.core.MoveReplicaHDFSTest_failed_coll_true.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2856665 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:46819/data
[junit4] 2> 2856665 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 2856666 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2856666 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-001/node3/MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2], dataDir=[hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node5/data/]
[junit4] 2> 2856675 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node5/data/snapshot_metadata
[junit4] 2> 2856679 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2856679 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1' using configuration from collection MoveReplicaHDFSTest_failed_coll_true, trusted=true
[junit4] 2> 2856680 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43887.solr.core.MoveReplicaHDFSTest_failed_coll_true.shard1.replica_n1' (registry 'solr.core.MoveReplicaHDFSTest_failed_coll_true.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11504f69
[junit4] 2> 2856680 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory solr.hdfs.home=hdfs://lucene2-us-west.apache.org:46819/data
[junit4] 2> 2856680 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory Solr Kerberos Authentication disabled
[junit4] 2> 2856680 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2856680 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.SolrCore [[MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J2/temp/solr.cloud.MoveReplicaHDFSTest_D93B52828EC50BB8-001/tempDir-001/node1/MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1], dataDir=[hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node3/data/]
[junit4] 2> 2856681 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node3/data/snapshot_metadata
[junit4] 2> 2856727 WARN (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 2856742 WARN (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 2856742 WARN (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 2856746 WARN (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.s.h.HdfsDirectory The NameNode is in SafeMode - Solr will wait 5 seconds and try again.
[junit4] 2> 2862163 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node7/data
[junit4] 2> 2862188 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node3/data
[junit4] 2> 2862195 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node8/data
[junit4] 2> 2862201 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node5/data
[junit4] 2> 2862261 INFO (qtp1272766324-22800) [n:127.0.0.1:42851_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node7 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n4] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node7/data/index
[junit4] 2> 2862286 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node5/data/index
[junit4] 2> 2862286 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node3/data/index
[junit4] 2> 2862304 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.HdfsDirectoryFactory creating directory factory for path hdfs://lucene2-us-west.apache.org:46819/data/MoveReplicaHDFSTest_failed_coll_true/core_node8/data/index
[junit4] 2> 2862676 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40655 is added to blk_1073741826_1002{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-eb984553-996c-4b9b-b087-af607130edec:NORMAL:127.0.0.1:34546|RBW], ReplicaUC[[DISK]DS-14749d85-1b31-40d5-a4e6-eba358d92531:NORMAL:127.0.0.1:40655|RBW]]} size 0
[junit4] 2> 2862676 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40655 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-eb984553-996c-4b9b-b087-af607130edec:NORMAL:127.0.0.1:34546|RBW], ReplicaUC[[DISK]DS-9dafaf1f-cce1-4816-acff-7f3923c3c568:NORMAL:127.0.0.1:40655|RBW]]} size 0
[junit4] 2> 2862690 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34546 is added to blk_1073741825_1001{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-eb984553-996c-4b9b-b087-af607130edec:NORMAL:127.0.0.1:34546|RBW], ReplicaUC[[DISK]DS-9dafaf1f-cce1-4816-acff-7f3923c3c568:NORMAL:127.0.0.1:40655|RBW]]} size 0
[junit4] 2> 2862718 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40655 is added to blk_1073741828_1004{UCState=COMMITTED, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-14749d85-1b31-40d5-a4e6-eba358d92531:NORMAL:127.0.0.1:40655|RBW], ReplicaUC[[DISK]DS-eb984553-996c-4b9b-b087-af607130edec:NORMAL:127.0.0.1:34546|RBW]]} size 69
[junit4] 2> 2862718 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34546 is added to blk_1073741828_1004 size 69
[junit4] 2> 2862718 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34546 is added to blk_1073741826_1002 size 69
[junit4] 2> 2862719 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:34546 is added to blk_1073741827_1003{UCState=UNDER_CONSTRUCTION, truncateBlock=null, primaryNodeIndex=-1, replicas=[ReplicaUC[[DISK]DS-9dafaf1f-cce1-4816-acff-7f3923c3c568:NORMAL:127.0.0.1:40655|RBW], ReplicaUC[[DISK]DS-4bf5ee87-9bf2-4247-81f0-6922a233962a:NORMAL:127.0.0.1:34546|RBW]]} size 0
[junit4] 2> 2862721 INFO (Block report processor) [ ] BlockStateChange BLOCK* addStoredBlock: blockMap updated: 127.0.0.1:40655 is added to blk_1073741827_1003 size 69
[junit4] 2> 2862975 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 2862975 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 2862975 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2862975 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2862975 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 2862975 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 2863019 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2863019 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2863033 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.HdfsUpdateLog
[junit4] 2> 2863033 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2863033 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.HdfsUpdateLog Initializing HdfsUpdateLog: tlogDfsReplication=3
[junit4] 2> 2863039 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2863039 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2863145 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2863145 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2863187 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.s.SolrIndexSearcher Opening [Searcher@16e43fe2[MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] main]
[junit4] 2> 2863205 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2863205 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2863206 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2863210 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@a3353de[MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] main]
[junit4] 2> 2863211 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2863211 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2863228 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2863238 INFO (searcherExecutor-8227-thread-1-processing-n:127.0.0.1:43887_solr x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1 c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.SolrCore [MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] Registered new searcher Searcher@a3353de[MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2863238 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1597551519873368064
[junit4] 2> 2863248 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1597551519883853824
[junit4] 2> 2863252 INFO (searcherExecutor-8224-thread-1-processing-n:127.0.0.1:43008_solr x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6 c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.SolrCore [MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] Registered new searcher Searcher@16e43fe2[MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2863288 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.ZkShardTerms Successful update of terms at /collections/MoveReplicaHDFSTest_failed_coll_true/terms/shard2 to Terms{values={core_node8=0}, version=0}
[junit4] 2> 2863304 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@b4f36b9[MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] main]
[junit4] 2> 2863305 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2863306 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2863306 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2863307 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1597551519945719808
[junit4] 2> 2863321 INFO (searcherExecutor-8226-thread-1-processing-n:127.0.0.1:37177_solr x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2 c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.SolrCore [MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] Registered new searcher Searcher@b4f36b9[MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2863323 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/MoveReplicaHDFSTest_failed_coll_true/terms/shard1 to Terms{values={core_node5=0}, version=0}
[junit4] 2> 2863342 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
[junit4] 2> 2863344 INFO (qtp2003698812-22801) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/MoveReplicaHDFSTest_failed_coll_true/terms/shard1 to Terms{values={core_node3=0, core_node5=0}, version=1}
[junit4] 2> 2863379 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2863379 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2863379 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37177/solr/MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2/
[junit4] 2> 2863379 INFO (qtp1603064068-22806) [n:127.0.0.1:37177_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node5 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n2 url=http://127.0.0.1:37177/solr START replicas=[http://127.0.0.1:43887/solr/MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1/] nUpdates=100
[junit4] 2> 2863380 INFO (qtp1737811792-23071) [n:127.0.0.1:43008_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard2 r:core_node8 x:MoveReplicaHDFSTest_failed_coll_true_shard2_replica_n6] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2863402 INFO (zkCallback-7399-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2863402 INFO (zkCallback-7399-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2863418 INFO (zkCallback-7354-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2863418 INFO (zkCallback-7354-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2863418 INFO (qtp2003698812-22810) [n:127.0.0.1:43887_solr c:MoveReplicaHDFSTest_failed_coll_true s:shard1 r:core_node3 x:MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] o.a.s.c.S.Request [MoveReplicaHDFSTest_failed_coll_true_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=35
[junit4] 2> 2863418 INFO (zkCallback-7350-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2863419 INFO (zkCallback-7352-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2863419 INFO (zkCallback-7350-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size: [5])
[junit4] 2> 2863419 INFO (zkCallback-7352-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MoveReplicaHDFSTest_failed_coll_true/state.json] for collection [MoveReplicaHDFSTest_failed_coll_true] has occurred - updating... (live nodes size
[...truncated too long message...]
oreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
[junit4] 2> at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
[junit4] 2> at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 19) Thread[id=4674, name=zkConnectionManagerCallback-1007-thread-1, state=WAITING, group=TGRP-HttpPartitionTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 20) Thread[id=4618, name=Scheduler-1483861829, state=TIMED_WAITING, group=TGRP-HttpPartitionTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 21) Thread[id=4679, name=Scheduler-96649350, state=WAITING, group=TGRP-HttpPartitionTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 22) Thread[id=4636, name=zkConnectionManagerCallback-992-thread-1, state=WAITING, group=TGRP-HttpPartitionTest]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
[junit4] 2> at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 23) Thread[id=4615, name=qtp1806234425-4615, state=RUNNABLE, group=TGRP-HttpPartitionTest]
[junit4] 2> at sun.nio.ch.IOUtil.drain(Native Method)
[junit4] 2> at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:104)
[junit4] 2> at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
[junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)
[junit4] 2> at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101)
[junit4] 2> at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375)
[junit4] 2> at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
[junit4] 2> at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=489, maxMBSortInHeap=6.10878187238618, sim=RandomSimilarity(queryNorm=false): {}, locale=mt, timezone=US/Pacific-New
[junit4] 2> NOTE: Linux 4.4.0-112-generic amd64/Oracle Corporation 1.8.0_152 (64-bit)/cpus=4,threads=43,free=264313616,total=472383488
[junit4] 2> NOTE: All tests run in this JVM: [FullSolrCloudDistribCmdsTest, TestHdfsUpdateLog, TestSolrCoreSnapshots, ChaosMonkeyNothingIsSafeTest, TestEmbeddedSolrServerConstructors, ComputePlanActionTest, TestCoreBackup, HighlighterConfigTest, TestFastOutputStream, TestLegacyFieldCache, TestDocSet, TestTriggerIntegration, TestPhraseSuggestions, TestUninvertingReader, DistributedFacetExistsSmallTest, TestBinaryResponseWriter, RAMDirectoryFactoryTest, TestComponentsName, HdfsThreadLeakTest, SolrPluginUtilsTest, TestUseDocValuesAsStored2, HdfsDirectoryFactoryTest, DebugComponentTest, TestMergePolicyConfig, TestSolrQueryParser, ConfigSetsAPITest, TestLuceneMatchVersion, TestFieldCache, DistributedQueryComponentCustomSortTest, RollingRestartTest, TestDownShardTolerantSearch, AtomicUpdateProcessorFactoryTest, HdfsCollectionsAPIDistributedZkTest, DistributedQueryElevationComponentTest, TestPHPSerializedResponseWriter, DocValuesNotIndexedTest, MetricsHandlerTest, CleanupOldIndexTest, TestRandomRequestDistribution, TestGenericDistributedQueue, HdfsBasicDistributedZk2Test, PreAnalyzedUpdateProcessorTest, TestReloadAndDeleteDocs, TestManagedResource, SegmentsInfoRequestHandlerTest, CollectionReloadTest, SolrIndexSplitterTest, TestSearcherReuse, SuggestComponentTest, TestCollationField, HttpPartitionTest]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HttpPartitionTest -Dtests.seed=D93B52828EC50BB8 -Dtests.multiplier=2 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=mt -Dtests.timezone=US/Pacific-New -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J0 | HttpPartitionTest (suite) <<<
[junit4] > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
[junit4] > at __randomizedtesting.SeedInfo.seed([D93B52828EC50BB8]:0)
[junit4] Completed [720/801 (2!)] on J0 in 7283.71s, 1 test, 2 errors <<< FAILURES!
[...truncated 38279 lines...]
-ecj-javadoc-lint-tests:
[mkdir] Created dir: /tmp/ecj331236023
[ecj-lint] Compiling 20 source files to /tmp/ecj331236023
[ecj-lint] ----------
[ecj-lint] 1. ERROR in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/spatial3d/src/test/org/apache/lucene/spatial3d/geom/RandomGeoPolygonTest.java (at line 27)
[ecj-lint] import static com.carrotsearch.randomizedtesting.RandomizedTest.randomDouble;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import com.carrotsearch.randomizedtesting.RandomizedTest.randomDouble is never used
[ecj-lint] ----------
[ecj-lint] 1 problem (1 error)
BUILD FAILED
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/build.xml:642: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/build.xml:101: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/build.xml:208: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/common-build.xml:2264: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/common-build.xml:2095: The following error occurred while executing this line:
/home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/common-build.xml:2128: Compile failed; see the compiler error output for details.
Total time: 222 minutes 37 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 43 - Still
unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/43/
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([F4FDB65561595C5B:A9B5A8DCAE9FFA14]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testSearchRate(TestTriggerIntegration.java:1199)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventFromRestoredState
Error Message:
The trigger did not fire at all
Stack Trace:
java.lang.AssertionError: The trigger did not fire at all
at __randomizedtesting.SeedInfo.seed([F4FDB65561595C5B:F4CB02954854F831]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration.testEventFromRestoredState(TestTriggerIntegration.java:691)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12854 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration
[junit4] 2> 592084 INFO (SUITE-TestTriggerIntegration-seed#[F4FDB65561595C5B]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestTriggerIntegration_F4FDB65561595C5B-001/init-core-data-001
[junit4] 2> 592086 WARN (SUITE-TestTriggerIntegration-seed#[F4FDB65561595C5B]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=42 numCloses=42
[junit4] 2> 592086 INFO (SUITE-TestTriggerIntegration-seed#[F4FDB65561595C5B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 592087 INFO (SUITE-TestTriggerIntegration-seed#[F4FDB65561595C5B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 592106 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeAddedTriggerRestoreState
[junit4] 2> 592110 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 592110 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 592110 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 592110 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 592110 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 592110 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 592110 DEBUG (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 592119 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 592120 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 1
[junit4] 2> 592120 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion -1
[junit4] 2> 592120 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 1
[junit4] 2> 592134 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10008_solr, 127.0.0.1:10007_solr]
[junit4] 2> 592146 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 592146 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 1, lastZnodeVersion 1
[junit4] 2> 592148 DEBUG (ScheduledTrigger-1619-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592171 DEBUG (ScheduledTrigger-1619-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592191 DEBUG (ScheduledTrigger-1619-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592211 DEBUG (ScheduledTrigger-1619-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592234 DEBUG (ScheduledTrigger-1619-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592247 DEBUG (simCloudManagerPool-1618-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 592247 DEBUG (simCloudManagerPool-1618-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 2
[junit4] 2> 592256 DEBUG (ScheduledTrigger-1619-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592257 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 2
[junit4] 2> 592257 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 592257 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 2, lastZnodeVersion 2
[junit4] 2> 592276 DEBUG (ScheduledTrigger-1619-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592283 DEBUG (simCloudManagerPool-1618-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 592283 DEBUG (simCloudManagerPool-1618-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 3
[junit4] 2> 592284 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 3
[junit4] 2> 592284 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 592284 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - Initial livenodes: [127.0.0.1:10009_solr, 127.0.0.1:10008_solr, 127.0.0.1:10007_solr]
[junit4] 2> 592284 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=5, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 592284 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger Adding node from marker path: 127.0.0.1:10009_solr
[junit4] 2> 592284 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10009_solr
[junit4] 2> 592285 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 3, lastZnodeVersion 3
[junit4] 2> 592285 DEBUG (ScheduledTrigger-1619-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 592285 DEBUG (ScheduledTrigger-1619-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 592297 DEBUG (ScheduledTrigger-1619-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 592305 DEBUG (ScheduledTrigger-1619-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 592305 DEBUG (ScheduledTrigger-1619-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 592318 DEBUG (ScheduledTrigger-1619-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 592325 DEBUG (ScheduledTrigger-1619-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 592325 DEBUG (ScheduledTrigger-1619-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 592338 DEBUG (ScheduledTrigger-1619-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 592357 DEBUG (ScheduledTrigger-1619-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 592357 DEBUG (ScheduledTrigger-1619-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 592357 DEBUG (simCloudManagerPool-1618-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 592358 DEBUG (simCloudManagerPool-1618-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 4
[junit4] 2> 592358 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 4
[junit4] 2> 592358 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 592358 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - Initial livenodes: [127.0.0.1:10009_solr, 127.0.0.1:10008_solr, 127.0.0.1:10007_solr]
[junit4] 2> 592359 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger instantiated with properties: {event=nodeAdded, waitFor=0, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}], enabled=true}
[junit4] 2> 592359 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 4, lastZnodeVersion 4
[junit4] 2> 592359 DEBUG (ScheduledTrigger-1619-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 592359 DEBUG (ScheduledTrigger-1619-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_restore_trigger
[junit4] 2> 592359 DEBUG (ScheduledTrigger-1619-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 592359 DEBUG (ScheduledTrigger-1619-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger firing registered processor for nodes: [127.0.0.1:10009_solr] added at times [24361529039765546], now=24361532761714146
[junit4] 2> 592359 DEBUG (ScheduledTrigger-1619-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"568cae7c99c02aTbecttotktkl746gglgm51fi0x",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":24361529039765546,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361529039765546],
[junit4] 2> "nodeNames":["127.0.0.1:10009_solr"]}}
[junit4] 2> 592359 DEBUG (ScheduledTrigger-1619-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_restore_trigger]
[junit4] 2> 592414 DEBUG (ScheduledTrigger-1619-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_restore_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10009_solr
[junit4] 2> 592415 DEBUG (AutoscalingActionExecutor-1620-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"568cae7c99c02aTbecttotktkl746gglgm51fi0x",
[junit4] 2> "source":"node_added_restore_trigger",
[junit4] 2> "eventTime":24361529039765546,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361529039765546],
[junit4] 2> "_enqueue_time_":24361532788546946,
[junit4] 2> "nodeNames":["127.0.0.1:10009_solr"]}}
[junit4] 2> 592416 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeAddedTriggerRestoreState
[junit4] 2> 592416 INFO (TEST-TestTriggerIntegration.testNodeAddedTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 3
[junit4] 2> ## Empty nodes: 3
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: []
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_restore_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2>
[junit4] 2> 592420 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeMarkersRegistration
[junit4] 2> 592420 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 5
[junit4] 2> 592420 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 592420 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 592421 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 592421 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 592421 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 592421 WARN (AutoscalingActionExecutor-1620-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> org.apache.lucene.store.AlreadyClosedException: ScheduledTrigger node_added_restore_trigger has been closed.
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.dequeue(ScheduledTriggers.java:546) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:319) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_152]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_152]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> 592422 DEBUG (AutoscalingActionExecutor-1620-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 374 ms for event id=568cae7c99c02aTbecttotktkl746gglgm51fi0x
[junit4] 2> 592422 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 592422 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 592422 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 592438 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 592439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 6
[junit4] 2> 592439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion -1
[junit4] 2> 592439 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 6
[junit4] 2> 592452 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10009_solr, 127.0.0.1:10008_solr]
[junit4] 2> 592466 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 592466 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 6, lastZnodeVersion 6
[junit4] 2> 592467 DEBUG (ScheduledTrigger-1622-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592500 DEBUG (ScheduledTrigger-1622-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592520 DEBUG (ScheduledTrigger-1622-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592550 DEBUG (ScheduledTrigger-1622-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592552 DEBUG (simCloudManagerPool-1621-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 592552 DEBUG (simCloudManagerPool-1621-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 592553 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 592553 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 592553 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 592554 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== KILL OVERSEER 1
[junit4] 2> 592554 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 592554 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 592554 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 592554 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 592554 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 592554 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 592555 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 592555 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 592728 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 7
[junit4] 2> 592729 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion -1
[junit4] 2> 592729 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 7
[junit4] 2> 592729 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10008_solr]
[junit4] 2> 592729 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10009_solr
[junit4] 2> 592738 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 592738 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 7, lastZnodeVersion 7
[junit4] 2> 592738 DEBUG (ScheduledTrigger-1625-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592738 DEBUG (ScheduledTrigger-1625-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10009_solr
[junit4] 2> 592759 DEBUG (ScheduledTrigger-1625-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592780 DEBUG (ScheduledTrigger-1625-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592801 DEBUG (ScheduledTrigger-1625-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592821 DEBUG (ScheduledTrigger-1625-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592830 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== ADD TRIGGERS
[junit4] 2> 592841 DEBUG (ScheduledTrigger-1625-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592859 INFO (simCloudManagerPool-1624-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 592859 DEBUG (simCloudManagerPool-1624-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 592859 DEBUG (simCloudManagerPool-1624-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 8
[junit4] 2> 592860 INFO (simCloudManagerPool-1624-thread-1) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 592860 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 8
[junit4] 2> 592860 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10008_solr]
[junit4] 2> 592860 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventMarkerAction}], enabled=true}
[junit4] 2> 592860 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 8, lastZnodeVersion 8
[junit4] 2> 592861 DEBUG (ScheduledTrigger-1625-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 592861 DEBUG (ScheduledTrigger-1625-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 592861 DEBUG (ScheduledTrigger-1625-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 592879 INFO (simCloudManagerPool-1624-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 592879 DEBUG (simCloudManagerPool-1624-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 592879 DEBUG (simCloudManagerPool-1624-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 592880 INFO (simCloudManagerPool-1624-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 592880 INFO (simCloudManagerPool-1624-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 592880 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 592881 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10008_solr]
[junit4] 2> 592881 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 592881 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== ADD NODE 1
[junit4] 2> 592881 DEBUG (ScheduledTrigger-1625-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 592881 DEBUG (ScheduledTrigger-1625-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 592881 DEBUG (ScheduledTrigger-1625-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 592881 DEBUG (ScheduledTrigger-1625-thread-4) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10011_solr at time 24361558888004696
[junit4] 2> 592881 DEBUG (ScheduledTrigger-1625-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 592901 DEBUG (ScheduledTrigger-1625-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 3
[junit4] 2> 592902 DEBUG (ScheduledTrigger-1625-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 592902 DEBUG (ScheduledTrigger-1625-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 592902 DEBUG (ScheduledTrigger-1625-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10011_solr] added at times [24361558888004696], now=24361559904832346
[junit4] 2> 592902 DEBUG (ScheduledTrigger-1625-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"568cb56fb1bc58Tbecttotktkl746gglgm51fi0z",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":24361558888004696,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361558888004696],
[junit4] 2> "nodeNames":["127.0.0.1:10011_solr"]}}
[junit4] 2> 592906 DEBUG (ScheduledTrigger-1625-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger, node_added_trigger]
[junit4] 2> 592931 DEBUG (ScheduledTrigger-1625-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10011_solr
[junit4] 2> 592931 DEBUG (AutoscalingActionExecutor-1626-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"568cb56fb1bc58Tbecttotktkl746gglgm51fi0z",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":24361558888004696,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361558888004696],
[junit4] 2> "_enqueue_time_":24361560126787446,
[junit4] 2> "nodeNames":["127.0.0.1:10011_solr"]}}
[junit4] 2> 592932 DEBUG (AutoscalingActionExecutor-1626-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 592932 DEBUG (AutoscalingActionExecutor-1626-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
[junit4] 2> 592932 DEBUG (AutoscalingActionExecutor-1626-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 592932 DEBUG (AutoscalingActionExecutor-1626-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 51 ms for event id=568cb56fb1bc58Tbecttotktkl746gglgm51fi0z
[junit4] 2> 592982 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.TestTriggerIntegration ====== KILL OVERSEER 2
[junit4] 2> 592982 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 592982 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 592982 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 592982 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 592982 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 592982 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 592984 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 592984 DEBUG (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 592998 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 9
[junit4] 2> 592999 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 592999 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventMarkerAction init
[junit4] 2> 592999 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion -1
[junit4] 2> 592999 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 9
[junit4] 2> 592999 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_trigger - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10011_solr]
[junit4] 2> 592999 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10008_solr
[junit4] 2> 593018 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10011_solr]
[junit4] 2> 593018 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventMarkerAction}], enabled=true}
[junit4] 2> 593022 DEBUG (ScheduledTrigger-1628-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 593022 DEBUG (ScheduledTrigger-1628-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10008_solr
[junit4] 2> 593038 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10011_solr]
[junit4] 2> 593039 DEBUG (ScheduledTrigger-1628-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 593039 DEBUG (ScheduledTrigger-1628-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 593039 DEBUG (ScheduledTrigger-1628-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593039 DEBUG (ScheduledTrigger-1628-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10008_solr
[junit4] 2> 593050 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 9, lastZnodeVersion 9
[junit4] 2> 593065 DEBUG (ScheduledTrigger-1628-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 593065 DEBUG (ScheduledTrigger-1628-thread-2) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10008_solr]
[junit4] 2> 593065 DEBUG (ScheduledTrigger-1628-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"568cb713e089b6Tbecttotktkl746gglgm51fi12",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":24361565937502646,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361565937502646],
[junit4] 2> "nodeNames":["127.0.0.1:10008_solr"]}}
[junit4] 2> 593065 DEBUG (ScheduledTrigger-1628-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_trigger, node_added_trigger]
[junit4] 2> 593083 DEBUG (AutoscalingActionExecutor-1629-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"568cb713e089b6Tbecttotktkl746gglgm51fi12",
[junit4] 2> "source":"node_lost_trigger",
[junit4] 2> "eventTime":24361565937502646,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361565937502646],
[junit4] 2> "_enqueue_time_":24361568074777046,
[junit4] 2> "nodeNames":["127.0.0.1:10008_solr"]}}
[junit4] 2> 593089 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeMarkersRegistration
[junit4] 2> 593090 INFO (TEST-TestTriggerIntegration.testNodeMarkersRegistration-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 2
[junit4] 2> ## Dead nodes: 2
[junit4] 2> ## 127.0.0.1:10008_solr
[junit4] 2> ## 127.0.0.1:10009_solr
[junit4] 2> ## Collections: []
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 3
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2> ## * Trigger: node_lost_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2>
[junit4] 2> 593206 DEBUG (AutoscalingActionExecutor-1629-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 593206 DEBUG (AutoscalingActionExecutor-1629-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_lost_trigger after 100ms
[junit4] 2> 593206 DEBUG (AutoscalingActionExecutor-1629-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 593206 DEBUG (AutoscalingActionExecutor-1629-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 6186 ms for event id=568cb713e089b6Tbecttotktkl746gglgm51fi12
[junit4] 2> 593306 DEBUG (ScheduledTrigger-1628-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 593327 DEBUG (ScheduledTrigger-1628-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_trigger with currently live nodes: 2
[junit4] 2> 593330 DEBUG (ScheduledTrigger-1628-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593330 DEBUG (ScheduledTrigger-1628-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 593330 DEBUG (ScheduledTrigger-1628-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 593332 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testNodeLostTriggerRestoreState
[junit4] 2> 593332 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 10
[junit4] 2> 593332 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 10
[junit4] 2> 593332 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10009_solr
[junit4] 2> 593332 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10008_solr
[junit4] 2> 593332 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 593332 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 593332 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 593332 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 593332 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 593332 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 593333 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 593335 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 593335 DEBUG (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 593332 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireInterruptibly(AbstractQueuedSynchronizer.java:1220) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.locks.ReentrantLock.lockInterruptibly(ReentrantLock.java:335) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:169) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> 593344 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 593345 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 11
[junit4] 2> 593345 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion -1
[junit4] 2> 593345 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 11
[junit4] 2> 593374 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10011_solr]
[junit4] 2> 593382 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 593383 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 11, lastZnodeVersion 11
[junit4] 2> 593383 DEBUG (ScheduledTrigger-1631-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593403 DEBUG (ScheduledTrigger-1631-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593423 DEBUG (ScheduledTrigger-1631-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593444 DEBUG (ScheduledTrigger-1631-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593459 DEBUG (simCloudManagerPool-1630-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 593459 DEBUG (simCloudManagerPool-1630-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 12
[junit4] 2> 593460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 12
[junit4] 2> 593460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 593460 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 12, lastZnodeVersion 12
[junit4] 2> 593465 DEBUG (ScheduledTrigger-1631-thread-4) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 593475 DEBUG (simCloudManagerPool-1630-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 593475 DEBUG (simCloudManagerPool-1630-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 13
[junit4] 2> 593476 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 13
[junit4] 2> 593476 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 593476 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_restore_trigger - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10012_solr, 127.0.0.1:10011_solr]
[junit4] 2> 593476 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 593476 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 13, lastZnodeVersion 13
[junit4] 2> 593476 DEBUG (ScheduledTrigger-1631-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 3
[junit4] 2> 593485 DEBUG (ScheduledTrigger-1631-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593485 DEBUG (ScheduledTrigger-1631-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10012_solr
[junit4] 2> 593496 DEBUG (ScheduledTrigger-1631-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 593497 DEBUG (ScheduledTrigger-1631-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Tracking lost node: 127.0.0.1:10012_solr
[junit4] 2> 593505 DEBUG (ScheduledTrigger-1631-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593517 DEBUG (ScheduledTrigger-1631-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 593518 DEBUG (simCloudManagerPool-1630-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 593518 DEBUG (simCloudManagerPool-1630-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 14
[junit4] 2> 593518 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 14
[junit4] 2> 593522 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 593523 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger node_lost_restore_trigger - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10011_solr]
[junit4] 2> 593523 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger Adding lost node from marker path: 127.0.0.1:10012_solr
[junit4] 2> 593523 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 593523 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 14, lastZnodeVersion 14
[junit4] 2> 593523 DEBUG (ScheduledTrigger-1631-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: node_lost_restore_trigger with currently live nodes: 2
[junit4] 2> 593523 DEBUG (ScheduledTrigger-1631-thread-2) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger firing registered processor for lost nodes: [127.0.0.1:10012_solr]
[junit4] 2> 593523 DEBUG (ScheduledTrigger-1631-thread-2) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"568cbce6b72e62Tbecttotktkl746gglgm51fi15",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":24361590949621346,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361590949621346],
[junit4] 2> "nodeNames":["127.0.0.1:10012_solr"]}}
[junit4] 2> 593523 DEBUG (ScheduledTrigger-1631-thread-2) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_lost_restore_trigger]
[junit4] 2> 593535 DEBUG (AutoscalingActionExecutor-1632-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"568cbce6b72e62Tbecttotktkl746gglgm51fi15",
[junit4] 2> "source":"node_lost_restore_trigger",
[junit4] 2> "eventTime":24361590949621346,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361590949621346],
[junit4] 2> "_enqueue_time_":24361590980493396,
[junit4] 2> "nodeNames":["127.0.0.1:10012_solr"]}}
[junit4] 2> 593536 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testNodeLostTriggerRestoreState
[junit4] 2> 593536 INFO (TEST-TestTriggerIntegration.testNodeLostTriggerRestoreState-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 2
[junit4] 2> ## Empty nodes: 2
[junit4] 2> ## Dead nodes: 1
[junit4] 2> ## 127.0.0.1:10012_solr
[junit4] 2> ## Collections: []
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 3
[junit4] 2> ## - update 1
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_lost_restore_trigger
[junit4] 2> ## - STARTED 1
[junit4] 2>
[junit4] 2> 593561 INFO (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testListeners
[junit4] 2> 593561 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 15
[junit4] 2> 593561 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimNodeStateProvider - removing dead node values: 127.0.0.1:10012_solr
[junit4] 2> 593561 INFO (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 593562 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 593566 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 593566 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 593566 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 593567 WARN (AutoscalingActionExecutor-1632-thread-1) [ ] o.a.s.c.a.SystemLogListener Exception sending event to collection .system
[junit4] 2> java.io.IOException: java.lang.InterruptedException
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:540) ~[test/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.SystemLogListener.onEvent(SystemLogListener.java:118) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:790) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerListeners.fireListeners(ScheduledTriggers.java:757) ~[java/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:322) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_152]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_152]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> Caused by: java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.FutureTask.awaitDone(FutureTask.java:404) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.FutureTask.get(FutureTask.java:191) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:538) ~[test/:?]
[junit4] 2> ... 10 more
[junit4] 2> 593567 DEBUG (AutoscalingActionExecutor-1632-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 1596 ms for event id=568cbce6b72e62Tbecttotktkl746gglgm51fi15
[junit4] 2> 593568 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 593568 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 593568 DEBUG (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 593576 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 593576 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 16
[junit4] 2> 593576 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion -1
[junit4] 2> 593576 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 16
[junit4] 2> 593582 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10011_solr]
[junit4] 2> 593594 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 593594 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 16, lastZnodeVersion 16
[junit4] 2> 593595 DEBUG (ScheduledTrigger-1634-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593712 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593713 DEBUG (simCloudManagerPool-1633-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 593714 DEBUG (simCloudManagerPool-1633-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 17
[junit4] 2> 593719 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 17
[junit4] 2> 593719 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 593719 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 17, lastZnodeVersion 17
[junit4] 2> 593731 DEBUG (simCloudManagerPool-1633-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 593732 DEBUG (simCloudManagerPool-1633-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 18
[junit4] 2> 593732 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 18
[junit4] 2> 593732 INFO (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 593732 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10011_solr]
[junit4] 2> 593733 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger instantiated with properties: {event=nodeAdded, waitFor=2, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestTriggerAction}, {name=test1, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestDummyAction}], enabled=true}
[junit4] 2> 593733 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 18, lastZnodeVersion 18
[junit4] 2> 593733 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593733 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 593733 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 593745 DEBUG (simCloudManagerPool-1633-thread-3) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 593745 DEBUG (simCloudManagerPool-1633-thread-3) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 19
[junit4] 2> 593746 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 19
[junit4] 2> 593746 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 19, lastZnodeVersion 19
[junit4] 2> 593753 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 593753 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 593753 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 593754 DEBUG (simCloudManagerPool-1633-thread-4) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 593754 DEBUG (simCloudManagerPool-1633-thread-4) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 20
[junit4] 2> 593755 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 20
[junit4] 2> 593755 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 20, lastZnodeVersion 20
[junit4] 2> 593773 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 593773 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 593773 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 593773 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10013_solr at time 24361603492311396
[junit4] 2> 593794 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 593794 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 593794 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 593814 DEBUG (ScheduledTrigger-1634-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 593814 DEBUG (ScheduledTrigger-1634-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 593814 DEBUG (ScheduledTrigger-1634-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 593815 DEBUG (ScheduledTrigger-1634-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10013_solr] added at times [24361603492311396], now=24361605546146146
[junit4] 2> 593815 DEBUG (ScheduledTrigger-1634-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"568cbfd2517164Tbecttotktkl746gglgm51fi18",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":24361603492311396,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361603492311396],
[junit4] 2> "nodeNames":["127.0.0.1:10013_solr"]}}
[junit4] 2> 593815 DEBUG (ScheduledTrigger-1634-thread-3) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 593834 DEBUG (ScheduledTrigger-1634-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10013_solr
[junit4] 2> 593834 DEBUG (AutoscalingActionExecutor-1635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"568cbfd2517164Tbecttotktkl746gglgm51fi18",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":24361603492311396,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361603492311396],
[junit4] 2> "_enqueue_time_":24361605588831696,
[junit4] 2> "nodeNames":["127.0.0.1:10013_solr"]}}
[junit4] 2> 593854 DEBUG (AutoscalingActionExecutor-1635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 593854 DEBUG (AutoscalingActionExecutor-1635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 593854 DEBUG (AutoscalingActionExecutor-1635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 1001 ms for event id=568cbfd2517164Tbecttotktkl746gglgm51fi18
[junit4] 2> 593954 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 593954 DEBUG (ScheduledTrigger-1634-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 593955 DEBUG (ScheduledTrigger-1634-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 593955 DEBUG (ScheduledTrigger-1634-thread-2) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10014_solr at time 24361612551539496
[junit4] 2> 593975 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 593975 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 593975 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 593995 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 4
[junit4] 2> 593995 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger
[junit4] 2> 593995 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 4
[junit4] 2> 593995 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger firing registered processor for nodes: [127.0.0.1:10014_solr] added at times [24361612551539496], now=24361614590599346
[junit4] 2> 593996 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"568cc1ee4a4b28Tbecttotktkl746gglgm51fi1b",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":24361612551539496,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361612551539496],
[junit4] 2> "nodeNames":["127.0.0.1:10014_solr"]}}
[junit4] 2> 593996 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Pausing all triggers: [.auto_add_replicas, node_added_trigger]
[junit4] 2> 594006 DEBUG (ScheduledTrigger-1634-thread-1) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger - removing marker path: /autoscaling/nodeAdded/127.0.0.1:10014_solr
[junit4] 2> 594007 DEBUG (AutoscalingActionExecutor-1635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing actions for {
[junit4] 2> "id":"568cc1ee4a4b28Tbecttotktkl746gglgm51fi1b",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":24361612551539496,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361612551539496],
[junit4] 2> "_enqueue_time_":24361614604136746,
[junit4] 2> "nodeNames":["127.0.0.1:10014_solr"]}}
[junit4] 2> 594015 WARN (AutoscalingActionExecutor-1635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Exception executing actions
[junit4] 2> java.lang.Exception: Error executing action: test1 for trigger event: {
[junit4] 2> "id":"568cc1ee4a4b28Tbecttotktkl746gglgm51fi1b",
[junit4] 2> "source":"node_added_trigger",
[junit4] 2> "eventTime":24361612551539496,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361612551539496],
[junit4] 2> "_enqueue_time_":24361614604136746,
[junit4] 2> "nodeNames":["127.0.0.1:10014_solr"]}}
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:312) ~[java/:?]
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) ~[java/:?]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_152]
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_152]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> Caused by: java.lang.RuntimeException: failure
[junit4] 2> at org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestDummyAction.process(TestTriggerIntegration.java:909) ~[test/:?]
[junit4] 2> at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:309) ~[java/:?]
[junit4] 2> ... 6 more
[junit4] 2> 594016 DEBUG (AutoscalingActionExecutor-1635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: .auto_add_replicas after 100ms
[junit4] 2> 594016 DEBUG (AutoscalingActionExecutor-1635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers Resuming trigger: node_added_trigger after 100ms
[junit4] 2> 594016 DEBUG (AutoscalingActionExecutor-1635-thread-1) [ ] o.a.s.c.a.ScheduledTriggers -- processing took 471 ms for event id=568cc1ee4a4b28Tbecttotktkl746gglgm51fi1b
[junit4] 2> 594048 INFO (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testListeners
[junit4] 2> 594049 INFO (TEST-TestTriggerIntegration.testListeners-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimSolrCloudTestCase #######################################
[junit4] 2> ############ CLUSTER STATE ############
[junit4] 2> #######################################
[junit4] 2> ## Live nodes: 4
[junit4] 2> ## Empty nodes: 4
[junit4] 2> ## Dead nodes: 0
[junit4] 2> ## Collections: []
[junit4] 2> ## Max replicas per node: 0
[junit4] 2> ## Min replicas per node: 0
[junit4] 2> ## Total replicas: 0
[junit4] 2> ######### Solr op counts ##########
[junit4] 2> ## - autoscaling 4
[junit4] 2> ## - update 4
[junit4] 2> ######### Autoscaling event counts ###########
[junit4] 2> ## * Trigger: node_added_trigger
[junit4] 2> ## - FAILED 1
[junit4] 2> ## - STARTED 2
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2> ## * Trigger: node_lost_restore_trigger
[junit4] 2> ## - SUCCEEDED 1
[junit4] 2>
[junit4] 2> 594052 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[F4FDB65561595C5B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testEventQueue
[junit4] 2> 594053 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 21
[junit4] 2> 594053 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 21
[junit4] 2> 594053 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeLost markers
[junit4] 2> 594053 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 594053 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 21, lastZnodeVersion 21
[junit4] 2> 594053 INFO (TEST-TestTriggerIntegration.testEventQueue-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.s.SimCloudManager === Restarting OverseerTriggerThread and clearing object cache...
[junit4] 2> 594053 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Interrupted
[junit4] 2> java.lang.InterruptedException: null
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014) ~[?:1.8.0_152]
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048) ~[?:1.8.0_152]
[junit4] 2> at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:181) [java/:?]
[junit4] 2> at java.lang.Thread.run(Thread.java:748) [?:1.8.0_152]
[junit4] 2> 594053 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down scheduled thread pool executor now
[junit4] 2> 594054 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Shutting down action executor now
[junit4] 2> 594054 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for action executor
[junit4] 2> 594054 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers Awaiting termination for scheduled thread pool executor
[junit4] 2> 594054 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.ScheduledTriggers ScheduledTriggers closed completely
[junit4] 2> 594054 DEBUG (TEST-TestTriggerIntegration.testEventQueue-seed#[F4FDB65561595C5B]) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread has been closed explicitly
[junit4] 2> 594074 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Adding .auto_add_replicas and .scheduled_maintenance triggers
[junit4] 2> 594075 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 22
[junit4] 2> 594075 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 22, lastZnodeVersion -1
[junit4] 2> 594075 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 22
[junit4] 2> 594076 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeLostTrigger NodeLostTrigger .auto_add_replicas - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10013_solr]
[junit4] 2> 594111 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 594111 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 22, lastZnodeVersion 22
[junit4] 2> 594111 DEBUG (ScheduledTrigger-1637-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 594132 DEBUG (ScheduledTrigger-1637-thread-1) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 594153 DEBUG (ScheduledTrigger-1637-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 594173 DEBUG (ScheduledTrigger-1637-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 594191 DEBUG (simCloudManagerPool-1636-thread-1) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 594191 DEBUG (simCloudManagerPool-1636-thread-1) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 23
[junit4] 2> 594192 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 23
[junit4] 2> 594192 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread -- clean old nodeAdded markers
[junit4] 2> 594192 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 23, lastZnodeVersion 23
[junit4] 2> 594193 DEBUG (ScheduledTrigger-1637-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 2
[junit4] 2> 594194 INFO (simCloudManagerPool-1636-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 594194 DEBUG (simCloudManagerPool-1636-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 594194 DEBUG (simCloudManagerPool-1636-thread-2) [ ] o.a.s.c.a.AutoScalingHandler Verified autoscaling configuration
[junit4] 2> 594194 DEBUG (simCloudManagerPool-1636-thread-2) [ ] o.a.s.c.a.OverseerTriggerThread Refreshing /autoscaling.json with znode version 24
[junit4] 2> 594195 INFO (simCloudManagerPool-1636-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestEventQueueAction instantiated
[junit4] 2> 594195 DEBUG (simCloudManagerPool-1636-thread-2) [ ] o.a.s.c.a.s.TestTriggerIntegration TestTriggerAction init
[junit4] 2> 594195 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Processed trigger updates upto znodeVersion 24
[junit4] 2> 594195 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 - Initial livenodes: [127.0.0.1:10010_solr, 127.0.0.1:10013_solr]
[junit4] 2> 594195 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 instantiated with properties: {event=nodeAdded, waitFor=1, actions=[{name=test, class=org.apache.solr.cloud.autoscaling.sim.TestTriggerIntegration$TestEventQueueAction}], enabled=true}
[junit4] 2> 594195 DEBUG (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread Current znodeVersion 24, lastZnodeVersion 24
[junit4] 2> 594195 DEBUG (ScheduledTrigger-1637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 594195 DEBUG (ScheduledTrigger-1637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 2
[junit4] 2> 594213 DEBUG (ScheduledTrigger-1637-thread-2) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 594215 DEBUG (ScheduledTrigger-1637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 594215 DEBUG (ScheduledTrigger-1637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 594216 DEBUG (ScheduledTrigger-1637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Tracking new node: 127.0.0.1:10015_solr at time 24361625594293396
[junit4] 2> 594233 DEBUG (ScheduledTrigger-1637-thread-3) [ ] o.a.s.c.a.NodeLostTrigger Running NodeLostTrigger: .auto_add_replicas with currently live nodes: 3
[junit4] 2> 594236 DEBUG (ScheduledTrigger-1637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Running NodeAddedTrigger node_added_trigger1
[junit4] 2> 594236 DEBUG (ScheduledTrigger-1637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger Found livenodes: 3
[junit4] 2> 594236 DEBUG (ScheduledTrigger-1637-thread-3) [ ] o.a.s.c.a.NodeAddedTrigger NodeAddedTrigger node_added_trigger1 firing registered processor for nodes: [127.0.0.1:10015_solr] added at times [24361625594293396], now=24361626608471546
[junit4] 2> 594236 DEBUG (ScheduledTrigger-1637-thread-3) [ ] o.a.s.c.a.ScheduledTriggers ++++++++ Cooldown inactive - processing event: {
[junit4] 2> "id":"568cc4f7b2ec94Tbecttotktkl746gglgm51fi1e",
[junit4] 2> "source":"node_added_trigger1",
[junit4] 2> "eventTime":24361625594293396,
[junit4] 2> "eventType":"NODEADDED",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[24361625594293396],
[junit4] 2> "nodeNames":["127.0.0.1:10015_solr"]}}
[junit4] 2> 594236 DEBUG (ScheduledTrigger-1637-thread-3) [ ] o.a.s.c.a.S
[...truncated too long message...]
d-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1928908922
[copy] Copying 40 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1928908922
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null1928908922
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1459724795
[copy] Copying 246 files to /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1459724795
[delete] Deleting directory /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1459724795
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 89ms :: artifacts dl 10ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 126 minutes 2 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 42 - Failure
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/42/
All tests passed
Build Log:
[...truncated 62081 lines...]
-ecj-javadoc-lint-tests:
[mkdir] Created dir: /tmp/ecj1050644116
[ecj-lint] Compiling 895 source files to /tmp/ecj1050644116
[ecj-lint] invalid Class-Path header in manifest of jar file: /x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet/jars/org.restlet-2.3.0.jar
[ecj-lint] invalid Class-Path header in manifest of jar file: /x1/jenkins/.ivy2/cache/org.restlet.jee/org.restlet.ext.servlet/jars/org.restlet.ext.servlet-2.3.0.jar
[ecj-lint] ----------
[ecj-lint] 1. WARNING in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/core/src/test/org/apache/solr/analysis/TokenizerChainTest.java (at line 37)
[ecj-lint] TokenizerChain tokenizerChain = new TokenizerChain(
[ecj-lint] ^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'tokenizerChain' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 2. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 25)
[ecj-lint] import java.util.concurrent.TimeUnit;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import java.util.concurrent.TimeUnit is never used
[ecj-lint] ----------
[ecj-lint] 3. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 26)
[ecj-lint] import java.util.stream.Collectors;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import java.util.stream.Collectors is never used
[ecj-lint] ----------
[ecj-lint] 4. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 32)
[ecj-lint] import org.apache.solr.cloud.overseer.OverseerAction;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.apache.solr.cloud.overseer.OverseerAction is never used
[ecj-lint] ----------
[ecj-lint] 5. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 38)
[ecj-lint] import org.apache.solr.common.cloud.ZkNodeProps;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.apache.solr.common.cloud.ZkNodeProps is never used
[ecj-lint] ----------
[ecj-lint] 6. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 39)
[ecj-lint] import org.apache.solr.common.cloud.ZkStateReader;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.apache.solr.common.cloud.ZkStateReader is never used
[ecj-lint] ----------
[ecj-lint] 7. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/ForceLeaderTest.java (at line 41)
[ecj-lint] import org.apache.solr.common.util.Utils;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.apache.solr.common.util.Utils is never used
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 8. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/MoveReplicaTest.java (at line 25)
[ecj-lint] import java.util.HashSet;
[ecj-lint] ^^^^^^^^^^^^^^^^^
[ecj-lint] The import java.util.HashSet is never used
[ecj-lint] ----------
[ecj-lint] 9. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/MoveReplicaTest.java (at line 42)
[ecj-lint] import org.apache.solr.common.cloud.CollectionStateWatcher;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.apache.solr.common.cloud.CollectionStateWatcher is never used
[ecj-lint] ----------
[ecj-lint] 10. ERROR in /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/core/src/test/org/apache/solr/cloud/MoveReplicaTest.java (at line 46)
[ecj-lint] import org.apache.solr.common.cloud.ZkStateReaderAccessor;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] The import org.apache.solr.common.cloud.ZkStateReaderAccessor is never used
[ecj-lint] ----------
[ecj-lint] 10 problems (9 errors, 1 warning)
BUILD FAILED
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/build.xml:642: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/build.xml:101: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build.xml:690: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/common-build.xml:2095: The following error occurred while executing this line:
/x1/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/common-build.xml:2128: Compile failed; see the compiler error output for details.
Total time: 102 minutes 34 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 41 - Still
Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/41/
7 tests failed.
FAILED: org.apache.solr.cloud.TestRebalanceLeaders.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:36878
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:36878
at __randomizedtesting.SeedInfo.seed([4C785A114CCDD246:C42C65CBE231BFBE]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:425)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:341)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 44 more
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestRebalanceLeaders
Error Message:
98 threads leaked from SUITE scope at org.apache.solr.cloud.TestRebalanceLeaders: 1) Thread[id=11148, name=qtp1110176047-11148, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 2) Thread[id=11319, name=ScheduledTrigger-2864-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 3) Thread[id=11215, name=qtp2019149649-11215, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 4) Thread[id=11149, name=qtp1110176047-11149, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 5) Thread[id=11168, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 6) Thread[id=11212, name=searcherExecutor-2852-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 7) Thread[id=11158, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 8) Thread[id=11112, name=qtp1213989072-11112, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 9) Thread[id=11236, name=Thread-2409, state=WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1737) 10) Thread[id=11334, name=searcherExecutor-2872-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 11) Thread[id=11306, name=zkConnectionManagerCallback-3323-thread-1-EventThread, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 12) Thread[id=11164, name=zkConnectionManagerCallback-3308-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 13) Thread[id=11301, name=zkConnectionManagerCallback-3338-thread-1-SendThread(127.0.0.1:46496), state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054) 14) Thread[id=11128, name=zkConnectionManagerCallback-3293-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 15) Thread[id=11325, name=ScheduledTrigger-2864-thread-2, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 16) Thread[id=11227, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 17) Thread[id=11278, name=zkConnectionManagerCallback-3330-thread-1-EventThread, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 18) Thread[id=11142, name=searcherExecutor-2830-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 19) Thread[id=11153, name=Scheduler-1954250675, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 20) Thread[id=11171, name=zkConnectionManagerCallback-3315-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 21) Thread[id=11218, name=qtp2019149649-11218, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 22) Thread[id=11214, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 23) Thread[id=11242, name=Scheduler-399728715, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 24) Thread[id=11309, name=zkConnectionManagerCallback-3308-thread-1-EventThread, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 25) Thread[id=11187, name=qtp1757433187-11187, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 26) Thread[id=11183, name=qtp1757433187-11183, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 27) Thread[id=11185, name=qtp1757433187-11185, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 28) Thread[id=11223, name=Scheduler-1516335860, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 29) Thread[id=11114, name=qtp1213989072-11114, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 30) Thread[id=11182, name=qtp1757433187-11182-acceptor-0@6197bf3e-ServerConnector@722b7581{HTTP/1.1,[http/1.1]}{127.0.0.1:39705}, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:379) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:638) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 31) Thread[id=11135, name=zkConnectionManagerCallback-3300-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 32) Thread[id=11238, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 33) Thread[id=11228, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 34) Thread[id=11177, name=searcherExecutor-2841-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 35) Thread[id=11147, name=qtp1110176047-11147-acceptor-0@28ab1472-ServerConnector@5b102bee{HTTP/1.1,[http/1.1]}{127.0.0.1:44554}, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:379) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:638) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 36) Thread[id=11356, name=AutoscalingActionExecutor-2865-thread-1, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316) at org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:58) at org.apache.solr.cloud.autoscaling.ScheduledTriggers.waitForPendingTasks(ScheduledTriggers.java:393) at org.apache.solr.cloud.autoscaling.ScheduledTriggers.lambda$null$3(ScheduledTriggers.java:301) at org.apache.solr.cloud.autoscaling.ScheduledTriggers$$Lambda$491/964088217.run(Unknown Source) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192) at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$141/864623513.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 37) Thread[id=11188, name=Scheduler-929609296, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 38) Thread[id=11287, name=zkConnectionManagerCallback-3293-thread-1-SendThread(127.0.0.1:46496), state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 39) Thread[id=11111, name=qtp1213989072-11111-acceptor-0@64bf0eef-ServerConnector@5e9ba06f{HTTP/1.1,[http/1.1]}{127.0.0.1:41350}, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:379) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:638) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 40) Thread[id=11216, name=qtp2019149649-11216, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.apache.solr.cloud.OverseerTaskQueue$LatchWatcher.await(OverseerTaskQueue.java:158) at org.apache.solr.cloud.OverseerTaskQueue.offer(OverseerTaskQueue.java:208) at org.apache.solr.handler.admin.CollectionsHandler.sendToOCPQueue(CollectionsHandler.java:347) at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:257) at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:231) at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199) at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:736) at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:717) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:498) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:393) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:339) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637) at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595) at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564) at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:527) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132) at org.eclipse.jetty.server.Server.handle(Server.java:530) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102) at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.io.ManagedSelector$$Lambda$223/1158720033.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 41) Thread[id=11299, name=zkConnectionManagerCallback-3308-thread-1-SendThread(127.0.0.1:46496), state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 42) Thread[id=11117, name=Scheduler-2100102444, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 43) Thread[id=11136, name=Scheduler-909146469, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 44) Thread[id=11145, name=qtp1110176047-11145, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.io.ManagedSelector$$Lambda$223/1158720033.run(Unknown Source) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 45) Thread[id=11302, name=zkConnectionManagerCallback-3338-thread-1-EventThread, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 46) Thread[id=11144, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 47) Thread[id=11116, name=qtp1213989072-11116, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 48) Thread[id=11132, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 49) Thread[id=11222, name=qtp2019149649-11222, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 50) Thread[id=11288, name=zkConnectionManagerCallback-3293-thread-1-EventThread, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 51) Thread[id=11199, name=zkConnectionManagerCallback-3323-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 52) Thread[id=11150, name=qtp1110176047-11150, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 53) Thread[id=11186, name=qtp1757433187-11186, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 54) Thread[id=11312, name=OverseerAutoScalingTriggerThread-72467330987130911-127.0.0.1:39705_-n_0000000005, state=WAITING, group=Overseer autoscaling triggers] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at org.apache.solr.cloud.autoscaling.OverseerTriggerThread.run(OverseerTriggerThread.java:181) at java.lang.Thread.run(Thread.java:748) 55) Thread[id=11180, name=qtp1757433187-11180, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 56) Thread[id=11322, name=OverseerThreadFactory-2866-thread-1, state=WAITING, group=Overseer collection creation process.] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) at java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 57) Thread[id=11121, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 58) Thread[id=11110, name=qtp1213989072-11110, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 59) Thread[id=11206, name=zkConnectionManagerCallback-3330-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 60) Thread[id=11184, name=qtp1757433187-11184, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 61) Thread[id=11327, name=ScheduledTrigger-2864-thread-3, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 62) Thread[id=11220, name=qtp2019149649-11220, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 63) Thread[id=11217, name=qtp2019149649-11217-acceptor-0@14b36c30-ServerConnector@a50f4e0{HTTP/1.1,[http/1.1]}{127.0.0.1:36878}, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422) at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250) at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:379) at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:638) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 64) Thread[id=11308, name=zkConnectionManagerCallback-3315-thread-1-EventThread, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 65) Thread[id=11152, name=qtp1110176047-11152, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 66) Thread[id=11293, name=zkConnectionManagerCallback-3300-thread-1-SendThread(127.0.0.1:46496), state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 67) Thread[id=11219, name=qtp2019149649-11219, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 68) Thread[id=11113, name=qtp1213989072-11113, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 69) Thread[id=11151, name=qtp1110176047-11151, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 70) Thread[id=11311, name=OverseerCollectionConfigSetProcessor-72467330987130911-127.0.0.1:39705_-n_0000000005, state=TIMED_WAITING, group=Overseer collection creation process.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) at org.apache.solr.cloud.OverseerTaskProcessor.amILeader(OverseerTaskProcessor.java:392) at org.apache.solr.cloud.OverseerTaskProcessor.run(OverseerTaskProcessor.java:193) at java.lang.Thread.run(Thread.java:748) 71) Thread[id=11203, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 72) Thread[id=11173, name=Scheduler-90989771, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 73) Thread[id=11307, name=zkConnectionManagerCallback-3315-thread-1-SendThread(127.0.0.1:46496), state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 74) Thread[id=11109, name=qtp1213989072-11109, state=RUNNABLE, group=TGRP-TestRebalanceLeaders] at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:101) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.select(ManagedSelector.java:375) at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:304) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:179) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140) at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 75) Thread[id=11115, name=qtp1213989072-11115, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308) at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626) at java.lang.Thread.run(Thread.java:748) 76) Thread[id=11179, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 77) Thread[id=11241, name=zkConnectionManagerCallback-3345-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 78) Thread[id=11146, name=qtp1110176047-11146, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 79) Thread[id=11305, name=zkConnectionManagerCallback-3323-thread-1-SendThread(127.0.0.1:46496), state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 80) Thread[id=11234, name=zkConnectionManagerCallback-3338-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 81) Thread[id=11303, name=zkConnectionManagerCallback-3345-thread-1-SendThread(127.0.0.1:46496), state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054) 82) Thread[id=11304, name=zkConnectionManagerCallback-3345-thread-1-EventThread, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 83) Thread[id=11207, name=Scheduler-1698464292, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1081) at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 84) Thread[id=11277, name=zkConnectionManagerCallback-3330-thread-1-SendThread(127.0.0.1:46496), state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.zookeeper.client.StaticHostProvider.next(StaticHostProvider.java:105) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1000) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1063) 85) Thread[id=11221, name=qtp2019149649-11221, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 86) Thread[id=11310, name=OverseerStateUpdate-72467330987130911-127.0.0.1:39705_-n_0000000005, state=TIMED_WAITING, group=Overseer state updater.] at java.lang.Thread.sleep(Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.getData(SolrZkClient.java:340) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.amILeader(Overseer.java:415) at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:145) at java.lang.Thread.run(Thread.java:748) 87) Thread[id=11166, name=Thread-2391, state=WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1737) 88) Thread[id=11130, name=Thread-2382, state=WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1737) 89) Thread[id=11201, name=Thread-2400, state=WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Object.wait(Native Method) at java.lang.Object.wait(Object.java:502) at org.apache.solr.core.CloserThread.run(CoreContainer.java:1737) 90) Thread[id=11193, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 91) Thread[id=11157, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 92) Thread[id=11181, name=qtp1757433187-11181, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 93) Thread[id=11329, name=qtp2019149649-11329, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392) at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600) at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49) at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663) at java.lang.Thread.run(Thread.java:748) 94) Thread[id=11330, name=ScheduledTrigger-2864-thread-4, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.solr.common.cloud.ZkCmdExecutor.retryDelay(ZkCmdExecutor.java:111) at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:70) at org.apache.solr.common.cloud.SolrZkClient.exists(SolrZkClient.java:316) at org.apache.solr.client.solrj.impl.ZkDistribStateManager.hasData(ZkDistribStateManager.java:58) at org.apache.solr.cloud.autoscaling.NodeLostTrigger.removeMarker(NodeLostTrigger.java:182) at org.apache.solr.cloud.autoscaling.NodeLostTrigger.lambda$run$2(NodeLostTrigger.java:161) at org.apache.solr.cloud.autoscaling.NodeLostTrigger$$Lambda$492/1587465664.accept(Unknown Source) at java.util.ArrayList.forEach(ArrayList.java:1257) at org.apache.solr.cloud.autoscaling.NodeLostTrigger.run(NodeLostTrigger.java:159) at org.apache.solr.cloud.autoscaling.ScheduledTriggers$TriggerWrapper.run(ScheduledTriggers.java:589) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 95) Thread[id=11294, name=zkConnectionManagerCallback-3300-thread-1-EventThread, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502) 96) Thread[id=11192, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 97) Thread[id=11122, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders] at java.lang.Thread.sleep(Native Method) at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66) at java.lang.Thread.run(Thread.java:748) 98) Thread[id=11332, name=searcherExecutor-2867-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 98 threads leaked from SUITE scope at org.apache.solr.cloud.TestRebalanceLeaders:
1) Thread[id=11148, name=qtp1110176047-11148, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
2) Thread[id=11319, name=ScheduledTrigger-2864-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
3) Thread[id=11215, name=qtp2019149649-11215, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2163)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.reservedWait(ReservedThreadExecutor.java:308)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:373)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
at java.lang.Thread.run(Thread.java:748)
4) Thread[id=11149, name=qtp1110176047-11149, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663)
at java.lang.Thread.run(Thread.java:748)
5) Thread[id=11168, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
6) Thread[id=11212, name=searcherExecutor-2852-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
7) Thread[id=11158, name=Connection evictor, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders]
at java.lang.Thread.sleep(Native Method)
at org.apache.http.impl.client.IdleConnectionEvictor$1.run(IdleConnectionEvictor.java:66)
at java.lang.Thread.run(Thread.java:748)
8) Thread[id=11112, name=qtp1213989072-11112, state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:392)
at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:600)
at org.eclipse.jetty.util.thread.QueuedThreadPool.access$800(QueuedThreadPool.java:49)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:663)
at java.lang.Thread.run(Thread.java:748)
9) Thread[id=11236, name=Thread-2409, state=WAITING, group=TGRP-TestRebalanceLeaders]
at java.lang.Object.wait(Native Method)
at java.lang.Object.wait(Object.java:502)
at org.apache.solr.core.CloserThread.run(CoreContainer.java:1737)
10) Thread[id=11334, name=searcherExecutor-2872-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
11) Thread[id=11306, name=zkConnectionManagerCallback-3323-thread-1-EventThread, state=WAITING, group=TGRP-TestRebalanceLeaders]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:502)
12) Thread[id=11164, name=zkConnectionManagerCallback-3308-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
13) Thread[id=11301, name=zkConnectionManagerCallback-3338-thread-1-SendThread(127.0.0.1:46496), state=TIMED_WAITING, group=TGRP-TestRebalanceLeaders]
at java.lang.Thread.sleep(Native Method)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1054)
14) Thread[id=11128, name=zkConnectionManagerCallback-3293-thread-1, state=WAITING, group=TGRP-TestRebalanceLeaders]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
15) Thread[id=11325, name=ScheduledTrigger-2864-thread-2, state=WAITING, group=TGRP-TestRebalanceLeaders]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1088)
at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor
[...truncated too long message...]
re] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null705409752
[copy] Copying 40 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null705409752
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null705409752
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1324611724
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1324611724
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1324611724
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 49ms :: artifacts dl 21ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 197 minutes 17 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
[JENKINS] Lucene-Solr-BadApples-Tests-7.x - Build # 40 - Unstable
Posted by Apache Jenkins Server <je...@builds.apache.org>.
Build: https://builds.apache.org/job/Lucene-Solr-BadApples-Tests-7.x/40/
6 tests failed.
FAILED: org.apache.solr.cloud.LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud
Error Message:
Could not find collection : legacyFalse
Stack Trace:
org.apache.solr.common.SolrException: Could not find collection : legacyFalse
at __randomizedtesting.SeedInfo.seed([CE06C185C04F8F23:1F01330064400411]:0)
at org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:118)
at org.apache.solr.cloud.SolrCloudTestCase.getCollectionState(SolrCloudTestCase.java:256)
at org.apache.solr.cloud.LegacyCloudClusterPropTest.checkMandatoryProps(LegacyCloudClusterPropTest.java:154)
at org.apache.solr.cloud.LegacyCloudClusterPropTest.createAndTest(LegacyCloudClusterPropTest.java:91)
at org.apache.solr.cloud.LegacyCloudClusterPropTest.testCreateCollectionSwitchLegacyCloud(LegacyCloudClusterPropTest.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestPullReplica.testKillLeader
Error Message:
Replica core_node4 not up to date after 10 seconds expected:<1> but was:<0>
Stack Trace:
java.lang.AssertionError: Replica core_node4 not up to date after 10 seconds expected:<1> but was:<0>
at __randomizedtesting.SeedInfo.seed([CE06C185C04F8F23:87103531A2F41B75]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.TestPullReplica.waitForNumDocsInAllReplicas(TestPullReplica.java:541)
at org.apache.solr.cloud.TestPullReplica.doTestNoLeader(TestPullReplica.java:489)
at org.apache.solr.cloud.TestPullReplica.testKillLeader(TestPullReplica.java:308)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.TestTlogReplica.testRecovery
Error Message:
Can not find doc 8 in http://127.0.0.1:39694/solr
Stack Trace:
java.lang.AssertionError: Can not find doc 8 in http://127.0.0.1:39694/solr
at __randomizedtesting.SeedInfo.seed([CE06C185C04F8F23:FF6B829ED1F4584]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:889)
at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:603)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.testCollectionsAPI
Error Message:
Error from server at https://127.0.0.1:36918/solr/awhollynewcollection_0: No registered leader was found after waiting for 4000ms , collection: awhollynewcollection_0 slice: shard3 saw state=DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/9)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-bfffffff", "state":"active", "replicas":{ "core_node2":{ "core":"awhollynewcollection_0_shard1_replica_n1", "base_url":"https://127.0.0.1:40917/solr", "node_name":"127.0.0.1:40917_solr", "state":"down", "type":"NRT"}, "core_node4":{ "core":"awhollynewcollection_0_shard1_replica_n3", "base_url":"https://127.0.0.1:44279/solr", "node_name":"127.0.0.1:44279_solr", "state":"active", "type":"NRT", "leader":"true"}}}, "shard2":{ "range":"c0000000-ffffffff", "state":"active", "replicas":{ "core_node6":{ "core":"awhollynewcollection_0_shard2_replica_n5", "base_url":"https://127.0.0.1:44116/solr", "node_name":"127.0.0.1:44116_solr", "state":"active", "type":"NRT"}, "core_node8":{ "core":"awhollynewcollection_0_shard2_replica_n7", "base_url":"https://127.0.0.1:36918/solr", "node_name":"127.0.0.1:36918_solr", "state":"active", "type":"NRT", "leader":"true"}}}, "shard3":{ "range":"0-3fffffff", "state":"active", "replicas":{ "core_node10":{ "core":"awhollynewcollection_0_shard3_replica_n9", "base_url":"https://127.0.0.1:40917/solr", "node_name":"127.0.0.1:40917_solr", "state":"down", "type":"NRT"}, "core_node12":{ "core":"awhollynewcollection_0_shard3_replica_n11", "base_url":"https://127.0.0.1:44279/solr", "node_name":"127.0.0.1:44279_solr", "state":"active", "type":"NRT"}}}, "shard4":{ "range":"40000000-7fffffff", "state":"active", "replicas":{ "core_node14":{ "core":"awhollynewcollection_0_shard4_replica_n13", "base_url":"https://127.0.0.1:44116/solr", "node_name":"127.0.0.1:44116_solr", "state":"active", "type":"NRT"}, "core_node16":{ "core":"awhollynewcollection_0_shard4_replica_n15", "base_url":"https://127.0.0.1:36918/solr", "node_name":"127.0.0.1:36918_solr", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"3", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0"} with live_nodes=[127.0.0.1:44116_solr, 127.0.0.1:36918_solr, 127.0.0.1:40917_solr, 127.0.0.1:44279_solr]
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36918/solr/awhollynewcollection_0: No registered leader was found after waiting for 4000ms , collection: awhollynewcollection_0 slice: shard3 saw state=DocCollection(awhollynewcollection_0//collections/awhollynewcollection_0/state.json/9)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-bfffffff",
"state":"active",
"replicas":{
"core_node2":{
"core":"awhollynewcollection_0_shard1_replica_n1",
"base_url":"https://127.0.0.1:40917/solr",
"node_name":"127.0.0.1:40917_solr",
"state":"down",
"type":"NRT"},
"core_node4":{
"core":"awhollynewcollection_0_shard1_replica_n3",
"base_url":"https://127.0.0.1:44279/solr",
"node_name":"127.0.0.1:44279_solr",
"state":"active",
"type":"NRT",
"leader":"true"}}},
"shard2":{
"range":"c0000000-ffffffff",
"state":"active",
"replicas":{
"core_node6":{
"core":"awhollynewcollection_0_shard2_replica_n5",
"base_url":"https://127.0.0.1:44116/solr",
"node_name":"127.0.0.1:44116_solr",
"state":"active",
"type":"NRT"},
"core_node8":{
"core":"awhollynewcollection_0_shard2_replica_n7",
"base_url":"https://127.0.0.1:36918/solr",
"node_name":"127.0.0.1:36918_solr",
"state":"active",
"type":"NRT",
"leader":"true"}}},
"shard3":{
"range":"0-3fffffff",
"state":"active",
"replicas":{
"core_node10":{
"core":"awhollynewcollection_0_shard3_replica_n9",
"base_url":"https://127.0.0.1:40917/solr",
"node_name":"127.0.0.1:40917_solr",
"state":"down",
"type":"NRT"},
"core_node12":{
"core":"awhollynewcollection_0_shard3_replica_n11",
"base_url":"https://127.0.0.1:44279/solr",
"node_name":"127.0.0.1:44279_solr",
"state":"active",
"type":"NRT"}}},
"shard4":{
"range":"40000000-7fffffff",
"state":"active",
"replicas":{
"core_node14":{
"core":"awhollynewcollection_0_shard4_replica_n13",
"base_url":"https://127.0.0.1:44116/solr",
"node_name":"127.0.0.1:44116_solr",
"state":"active",
"type":"NRT"},
"core_node16":{
"core":"awhollynewcollection_0_shard4_replica_n15",
"base_url":"https://127.0.0.1:36918/solr",
"node_name":"127.0.0.1:36918_solr",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"3",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0"} with live_nodes=[127.0.0.1:44116_solr, 127.0.0.1:36918_solr, 127.0.0.1:40917_solr, 127.0.0.1:44279_solr]
at __randomizedtesting.SeedInfo.seed([CE06C185C04F8F23:8673B531C67CA0B6]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1106)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.cloud.api.collections.CollectionsAPIDistributedZkTest.testCollectionsAPI(CollectionsAPIDistributedZkTest.java:464)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete
Error Message:
Error from server at https://127.0.0.1:46373/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=utf-8"/> <title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title> </head> <body><h2>HTTP ERROR 404</h2> <p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason: <pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.8.v20171121</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at https://127.0.0.1:46373/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.8.v20171121</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([CE06C185C04F8F23:6DFC6F2047A76586]:0)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:551)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1015)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:886)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:948)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:819)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete(TestCollectionsAPIViaSolrCloudCluster.java:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:46373/solr/testcollection_shard1_replica_n2: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 404 Can not find: /solr/testcollection_shard1_replica_n2/update</title>
</head>
<body><h2>HTTP ERROR 404</h2>
<p>Problem accessing /solr/testcollection_shard1_replica_n2/update. Reason:
<pre> Can not find: /solr/testcollection_shard1_replica_n2/update</pre></p><hr><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.4.8.v20171121</a><hr/>
</body>
</html>
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.lambda$directUpdate$0(CloudSolrClient.java:528)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
FAILED: org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.testBasic
Error Message:
java.util.concurrent.ExecutionException: java.io.IOException: org.apache.solr.api.ApiBag$ExceptionWithErrObject: Error in command payload, errors: [{suspend-trigger={name=.scheduled_maintenance}, errorMessages=[No trigger exists with name: .scheduled_maintenance]}],
Stack Trace:
java.io.IOException: java.util.concurrent.ExecutionException: java.io.IOException: org.apache.solr.api.ApiBag$ExceptionWithErrObject: Error in command payload, errors: [{suspend-trigger={name=.scheduled_maintenance}, errorMessages=[No trigger exists with name: .scheduled_maintenance]}],
at __randomizedtesting.SeedInfo.seed([CE06C185C04F8F23:65FCDC901F93090D]:0)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:540)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager$1.request(SimCloudManager.java:414)
at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
at org.apache.solr.cloud.autoscaling.sim.TestLargeCluster.setupTest(TestLargeCluster.java:107)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:968)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: java.io.IOException: org.apache.solr.api.ApiBag$ExceptionWithErrObject: Error in command payload, errors: [{suspend-trigger={name=.scheduled_maintenance}, errorMessages=[No trigger exists with name: .scheduled_maintenance]}],
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.request(SimCloudManager.java:538)
... 41 more
Caused by: java.io.IOException: org.apache.solr.api.ApiBag$ExceptionWithErrObject: Error in command payload, errors: [{suspend-trigger={name=.scheduled_maintenance}, errorMessages=[No trigger exists with name: .scheduled_maintenance]}],
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simHandleSolrRequest(SimCloudManager.java:578)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.lambda$request$12(SimCloudManager.java:537)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:192)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
... 1 more
Caused by: org.apache.solr.api.ApiBag$ExceptionWithErrObject: Error in command payload, errors: [{suspend-trigger={name=.scheduled_maintenance}, errorMessages=[No trigger exists with name: .scheduled_maintenance]}],
at org.apache.solr.cloud.autoscaling.AutoScalingHandler.processOps(AutoScalingHandler.java:210)
at org.apache.solr.cloud.autoscaling.AutoScalingHandler.handleRequestBody(AutoScalingHandler.java:148)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:199)
at org.apache.solr.cloud.autoscaling.sim.SimCloudManager.simHandleSolrRequest(SimCloudManager.java:575)
... 6 more
Build Log:
[...truncated 12811 lines...]
[junit4] Suite: org.apache.solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster
[junit4] 2> Creating dataDir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster_CE06C185C04F8F23-001/init-core-data-001
[junit4] 2> 416235 WARN (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[CE06C185C04F8F23]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=5 numCloses=5
[junit4] 2> 416235 INFO (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[CE06C185C04F8F23]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 416237 INFO (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[CE06C185C04F8F23]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 416237 INFO (SUITE-TestCollectionsAPIViaSolrCloudCluster-seed#[CE06C185C04F8F23]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 416240 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 5 servers in /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster_CE06C185C04F8F23-001/tempDir-001
[junit4] 2> 416240 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 416240 INFO (Thread-1193) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 416240 INFO (Thread-1193) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 416242 ERROR (Thread-1193) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 416341 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.c.ZkTestServer start zk server on port:45285
[junit4] 2> 416346 INFO (zkConnectionManagerCallback-982-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416378 INFO (jetty-launcher-979-thread-2) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 416378 INFO (jetty-launcher-979-thread-3) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 416380 INFO (jetty-launcher-979-thread-1) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 416380 INFO (jetty-launcher-979-thread-4) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 416382 INFO (jetty-launcher-979-thread-5) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 416383 INFO (jetty-launcher-979-thread-5) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 416383 INFO (jetty-launcher-979-thread-5) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 416384 INFO (jetty-launcher-979-thread-5) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 416384 INFO (jetty-launcher-979-thread-5) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@23ce6e24{/solr,null,AVAILABLE}
[junit4] 2> 416385 INFO (jetty-launcher-979-thread-5) [ ] o.e.j.s.AbstractConnector Started ServerConnector@9b270ca{SSL,[ssl, http/1.1]}{127.0.0.1:43194}
[junit4] 2> 416385 INFO (jetty-launcher-979-thread-5) [ ] o.e.j.s.Server Started @416422ms
[junit4] 2> 416385 INFO (jetty-launcher-979-thread-5) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43194}
[junit4] 2> 416385 ERROR (jetty-launcher-979-thread-5) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 416385 INFO (jetty-launcher-979-thread-5) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 416385 INFO (jetty-launcher-979-thread-5) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 416385 INFO (jetty-launcher-979-thread-5) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 416385 INFO (jetty-launcher-979-thread-5) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 416385 INFO (jetty-launcher-979-thread-5) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-15T12:19:07.354Z
[junit4] 2> 416387 INFO (jetty-launcher-979-thread-4) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 416387 INFO (jetty-launcher-979-thread-4) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 416387 INFO (jetty-launcher-979-thread-4) [ ] o.e.j.s.session Scavenging every 660000ms
[junit4] 2> 416387 INFO (jetty-launcher-979-thread-4) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7eabff23{/solr,null,AVAILABLE}
[junit4] 2> 416388 INFO (jetty-launcher-979-thread-4) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1ba4845d{SSL,[ssl, http/1.1]}{127.0.0.1:37940}
[junit4] 2> 416388 INFO (jetty-launcher-979-thread-4) [ ] o.e.j.s.Server Started @416425ms
[junit4] 2> 416388 INFO (jetty-launcher-979-thread-4) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37940}
[junit4] 2> 416388 ERROR (jetty-launcher-979-thread-4) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 416388 INFO (jetty-launcher-979-thread-4) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 416388 INFO (jetty-launcher-979-thread-4) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 416388 INFO (jetty-launcher-979-thread-4) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 416388 INFO (jetty-launcher-979-thread-4) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 416388 INFO (jetty-launcher-979-thread-4) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-15T12:19:07.357Z
[junit4] 2> 416394 INFO (jetty-launcher-979-thread-3) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 416394 INFO (jetty-launcher-979-thread-3) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 416394 INFO (jetty-launcher-979-thread-3) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 416394 INFO (jetty-launcher-979-thread-2) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 416394 INFO (jetty-launcher-979-thread-2) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 416394 INFO (jetty-launcher-979-thread-2) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 416394 INFO (zkConnectionManagerCallback-986-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416394 INFO (jetty-launcher-979-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@58a23a0c{/solr,null,AVAILABLE}
[junit4] 2> 416395 INFO (jetty-launcher-979-thread-4) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 416398 INFO (jetty-launcher-979-thread-1) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 416398 INFO (jetty-launcher-979-thread-1) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 416398 INFO (jetty-launcher-979-thread-1) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 416398 INFO (jetty-launcher-979-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5a3f4a2c{/solr,null,AVAILABLE}
[junit4] 2> 416399 INFO (jetty-launcher-979-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7ce98e46{SSL,[ssl, http/1.1]}{127.0.0.1:44816}
[junit4] 2> 416399 INFO (jetty-launcher-979-thread-1) [ ] o.e.j.s.Server Started @416436ms
[junit4] 2> 416399 INFO (jetty-launcher-979-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44816}
[junit4] 2> 416399 ERROR (jetty-launcher-979-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 416399 INFO (jetty-launcher-979-thread-1) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 416399 INFO (jetty-launcher-979-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 416399 INFO (jetty-launcher-979-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 416399 INFO (jetty-launcher-979-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 416399 INFO (jetty-launcher-979-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-15T12:19:07.368Z
[junit4] 2> 416402 INFO (jetty-launcher-979-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@685bd767{/solr,null,AVAILABLE}
[junit4] 2> 416402 INFO (jetty-launcher-979-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5517f5d1{SSL,[ssl, http/1.1]}{127.0.0.1:35413}
[junit4] 2> 416402 INFO (jetty-launcher-979-thread-3) [ ] o.e.j.s.Server Started @416440ms
[junit4] 2> 416402 INFO (zkConnectionManagerCallback-984-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416402 INFO (jetty-launcher-979-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=35413}
[junit4] 2> 416403 ERROR (jetty-launcher-979-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 416403 INFO (jetty-launcher-979-thread-3) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 416403 INFO (jetty-launcher-979-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 416403 INFO (jetty-launcher-979-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 416403 INFO (jetty-launcher-979-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 416403 INFO (jetty-launcher-979-thread-5) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 416403 INFO (jetty-launcher-979-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-15T12:19:07.372Z
[junit4] 2> 416409 INFO (jetty-launcher-979-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@67b6b402{SSL,[ssl, http/1.1]}{127.0.0.1:37627}
[junit4] 2> 416409 INFO (jetty-launcher-979-thread-2) [ ] o.e.j.s.Server Started @416447ms
[junit4] 2> 416409 INFO (jetty-launcher-979-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37627}
[junit4] 2> 416409 ERROR (jetty-launcher-979-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 416409 INFO (jetty-launcher-979-thread-2) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 416409 INFO (jetty-launcher-979-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 416409 INFO (jetty-launcher-979-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 416409 INFO (jetty-launcher-979-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 416410 INFO (jetty-launcher-979-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-15T12:19:07.378Z
[junit4] 2> 416411 INFO (zkConnectionManagerCallback-992-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416411 INFO (zkConnectionManagerCallback-988-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416412 INFO (jetty-launcher-979-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 416413 INFO (jetty-launcher-979-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 416414 INFO (jetty-launcher-979-thread-5) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45285/solr
[junit4] 2> 416418 INFO (zkConnectionManagerCallback-990-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416428 INFO (jetty-launcher-979-thread-4) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45285/solr
[junit4] 2> 416428 INFO (zkConnectionManagerCallback-996-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416430 INFO (jetty-launcher-979-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 416430 INFO (zkConnectionManagerCallback-1000-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416431 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1016d8127b30006, likely client has closed socket
[junit4] 2> 416432 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1016d8127b30007, likely client has closed socket
[junit4] 2> 416434 INFO (jetty-launcher-979-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45285/solr
[junit4] 2> 416439 INFO (zkConnectionManagerCallback-1006-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416450 INFO (zkConnectionManagerCallback-1004-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416453 INFO (jetty-launcher-979-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45285/solr
[junit4] 2> 416454 INFO (jetty-launcher-979-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45285/solr
[junit4] 2> 416467 INFO (zkConnectionManagerCallback-1008-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416468 INFO (zkConnectionManagerCallback-1016-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416470 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1016d8127b3000b, likely client has closed socket
[junit4] 2> 416470 INFO (zkConnectionManagerCallback-1020-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416471 INFO (zkConnectionManagerCallback-1013-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416473 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn Unable to read additional data from client sessionid 0x1016d8127b3000d, likely client has closed socket
[junit4] 2> 416483 INFO (jetty-launcher-979-thread-4) [n:127.0.0.1:37940_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 416484 INFO (jetty-launcher-979-thread-1) [n:127.0.0.1:44816_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 416485 INFO (jetty-launcher-979-thread-1) [n:127.0.0.1:44816_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44816_solr
[junit4] 2> 416486 INFO (jetty-launcher-979-thread-1) [n:127.0.0.1:44816_solr ] o.a.s.c.Overseer Overseer (id=72459470498889740-127.0.0.1:44816_solr-n_0000000000) starting
[junit4] 2> 416495 INFO (zkConnectionManagerCallback-1022-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416496 INFO (jetty-launcher-979-thread-4) [n:127.0.0.1:37940_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37940_solr
[junit4] 2> 416509 INFO (zkCallback-1007-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 416509 INFO (zkConnectionManagerCallback-1035-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416510 INFO (jetty-launcher-979-thread-1) [n:127.0.0.1:44816_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 416512 INFO (jetty-launcher-979-thread-1) [n:127.0.0.1:44816_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45285/solr ready
[junit4] 2> 416512 INFO (jetty-launcher-979-thread-3) [n:127.0.0.1:35413_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 416514 INFO (jetty-launcher-979-thread-3) [n:127.0.0.1:35413_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 416516 INFO (jetty-launcher-979-thread-3) [n:127.0.0.1:35413_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 416516 INFO (jetty-launcher-979-thread-3) [n:127.0.0.1:35413_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:35413_solr
[junit4] 2> 416517 INFO (zkConnectionManagerCallback-1028-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416518 INFO (zkCallback-1007-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 416534 INFO (jetty-launcher-979-thread-2) [n:127.0.0.1:37627_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 416541 INFO (zkCallback-1019-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 416541 INFO (zkCallback-1034-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 416551 INFO (jetty-launcher-979-thread-2) [n:127.0.0.1:37627_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 416552 INFO (jetty-launcher-979-thread-1) [n:127.0.0.1:44816_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 416552 INFO (jetty-launcher-979-thread-1) [n:127.0.0.1:44816_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44816_solr
[junit4] 2> 416566 INFO (zkCallback-1021-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 416570 INFO (zkCallback-1019-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 416570 INFO (zkCallback-1007-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 416571 INFO (zkCallback-1021-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 416571 INFO (zkCallback-1034-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 416574 INFO (jetty-launcher-979-thread-2) [n:127.0.0.1:37627_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 416574 INFO (jetty-launcher-979-thread-2) [n:127.0.0.1:37627_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37627_solr
[junit4] 2> 416578 INFO (zkCallback-1021-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 416579 INFO (zkCallback-1034-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 416582 INFO (zkCallback-1007-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 416589 INFO (zkCallback-1027-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 416602 INFO (zkCallback-1019-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 416602 INFO (zkCallback-1027-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 416612 INFO (jetty-launcher-979-thread-5) [n:127.0.0.1:43194_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 416614 INFO (jetty-launcher-979-thread-5) [n:127.0.0.1:43194_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 416616 INFO (jetty-launcher-979-thread-5) [n:127.0.0.1:43194_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 416616 INFO (jetty-launcher-979-thread-5) [n:127.0.0.1:43194_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43194_solr
[junit4] 2> 416617 INFO (zkCallback-1021-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 416617 INFO (zkCallback-1007-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 416617 INFO (zkCallback-1034-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 416618 INFO (zkCallback-1019-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 416652 INFO (zkCallback-1027-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 416675 INFO (zkCallback-1005-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 416759 INFO (jetty-launcher-979-thread-3) [n:127.0.0.1:35413_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35413.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416766 INFO (jetty-launcher-979-thread-3) [n:127.0.0.1:35413_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35413.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416766 INFO (jetty-launcher-979-thread-3) [n:127.0.0.1:35413_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_35413.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416767 INFO (jetty-launcher-979-thread-3) [n:127.0.0.1:35413_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster_CE06C185C04F8F23-001/tempDir-001/node3/.
[junit4] 2> 416773 INFO (zkConnectionManagerCallback-1042-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416774 INFO (jetty-launcher-979-thread-3) [n:127.0.0.1:35413_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 416774 INFO (jetty-launcher-979-thread-3) [n:127.0.0.1:35413_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45285/solr ready
[junit4] 2> 416793 INFO (jetty-launcher-979-thread-1) [n:127.0.0.1:44816_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44816.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416803 INFO (jetty-launcher-979-thread-1) [n:127.0.0.1:44816_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44816.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416803 INFO (jetty-launcher-979-thread-1) [n:127.0.0.1:44816_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44816.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416804 INFO (jetty-launcher-979-thread-1) [n:127.0.0.1:44816_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster_CE06C185C04F8F23-001/tempDir-001/node1/.
[junit4] 2> 416861 INFO (jetty-launcher-979-thread-2) [n:127.0.0.1:37627_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37627.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416884 INFO (jetty-launcher-979-thread-2) [n:127.0.0.1:37627_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37627.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416885 INFO (jetty-launcher-979-thread-2) [n:127.0.0.1:37627_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37627.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416886 INFO (jetty-launcher-979-thread-2) [n:127.0.0.1:37627_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster_CE06C185C04F8F23-001/tempDir-001/node2/.
[junit4] 2> 416924 INFO (zkConnectionManagerCallback-1047-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 416925 INFO (jetty-launcher-979-thread-2) [n:127.0.0.1:37627_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 416926 INFO (jetty-launcher-979-thread-2) [n:127.0.0.1:37627_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45285/solr ready
[junit4] 2> 416963 INFO (jetty-launcher-979-thread-5) [n:127.0.0.1:43194_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43194.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416970 INFO (jetty-launcher-979-thread-5) [n:127.0.0.1:43194_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43194.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416970 INFO (jetty-launcher-979-thread-5) [n:127.0.0.1:43194_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43194.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416974 INFO (jetty-launcher-979-thread-5) [n:127.0.0.1:43194_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster_CE06C185C04F8F23-001/tempDir-001/node5/.
[junit4] 2> 416983 INFO (jetty-launcher-979-thread-4) [n:127.0.0.1:37940_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37940.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416991 INFO (jetty-launcher-979-thread-4) [n:127.0.0.1:37940_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37940.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416991 INFO (jetty-launcher-979-thread-4) [n:127.0.0.1:37940_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37940.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 416992 INFO (jetty-launcher-979-thread-4) [n:127.0.0.1:37940_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster_CE06C185C04F8F23-001/tempDir-001/node4/.
[junit4] 2> 417009 INFO (zkConnectionManagerCallback-1052-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 417010 INFO (jetty-launcher-979-thread-5) [n:127.0.0.1:43194_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 417011 INFO (jetty-launcher-979-thread-5) [n:127.0.0.1:43194_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45285/solr ready
[junit4] 2> 417037 INFO (zkConnectionManagerCallback-1057-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 417038 INFO (jetty-launcher-979-thread-4) [n:127.0.0.1:37940_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 417039 INFO (jetty-launcher-979-thread-4) [n:127.0.0.1:37940_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45285/solr ready
[junit4] 2> 417084 INFO (zkConnectionManagerCallback-1059-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 417090 INFO (zkConnectionManagerCallback-1064-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 417090 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 417091 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45285/solr ready
[junit4] 2> 417094 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCollectionCreateSearchDelete
[junit4] 2> 417096 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@5517f5d1{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 417096 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=622351640
[junit4] 2> 417096 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 417096 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@4322ef4c: rootName = solr_35413, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5f610a31
[junit4] 2> 417102 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 417102 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3286d1ce: rootName = solr_35413, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@540c433d
[junit4] 2> 417105 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 417105 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@51503353: rootName = solr_35413, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@51109815
[junit4] 2> 417110 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:35413_solr
[junit4] 2> 417111 INFO (zkCallback-1027-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 417112 INFO (zkCallback-1007-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 417116 INFO (zkCallback-1019-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 417116 INFO (zkCallback-1034-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 417116 INFO (zkCallback-1021-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 417116 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 417131 INFO (zkCallback-1005-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 417133 INFO (zkCallback-1051-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 417133 INFO (zkCallback-1063-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 417133 INFO (zkCallback-1041-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 417133 INFO (zkCallback-1056-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 417133 INFO (zkCallback-1046-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (5) -> (4)
[junit4] 2> 417139 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@58a23a0c{/solr,null,UNAVAILABLE}
[junit4] 2> 417140 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.e.j.s.session Stopped scavenging
[junit4] 2> 417146 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.e.j.s.Server jetty-9.4.8.v20171121, build timestamp: 2017-11-21T21:27:37Z, git hash: 82b8fb23f757335bb3329d540ce37a2a2615f0a8
[junit4] 2> 417214 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.e.j.s.session DefaultSessionIdManager workerName=node0
[junit4] 2> 417214 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.e.j.s.session No SessionScavenger set, using defaults
[junit4] 2> 417214 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.e.j.s.session Scavenging every 600000ms
[junit4] 2> 417218 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3dcc7d20{/solr,null,AVAILABLE}
[junit4] 2> 417218 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@d9af01e{SSL,[ssl, http/1.1]}{127.0.0.1:46373}
[junit4] 2> 417219 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.e.j.s.Server Started @417256ms
[junit4] 2> 417219 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46373}
[junit4] 2> 417219 ERROR (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 417226 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
[junit4] 2> 417226 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.4.0
[junit4] 2> 417226 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 417226 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 417226 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-04-15T12:19:08.195Z
[junit4] 2> 417232 INFO (zkConnectionManagerCallback-1066-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 417233 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 417243 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45285/solr
[junit4] 2> 417246 INFO (zkConnectionManagerCallback-1070-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 417253 INFO (zkConnectionManagerCallback-1072-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 417261 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [n:127.0.0.1:46373_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 417268 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [n:127.0.0.1:46373_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 417270 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [n:127.0.0.1:46373_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 417270 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [n:127.0.0.1:46373_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46373_solr
[junit4] 2> 417271 INFO (zkCallback-1051-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 417271 INFO (zkCallback-1063-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 417271 INFO (zkCallback-1027-thread-2) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 417271 INFO (zkCallback-1007-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 417271 INFO (zkCallback-1005-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 417271 INFO (zkCallback-1046-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 417274 INFO (zkCallback-1034-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 417274 INFO (zkCallback-1019-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 417274 INFO (zkCallback-1056-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 417290 INFO (zkCallback-1071-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 417433 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [n:127.0.0.1:46373_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46373.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 417441 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [n:127.0.0.1:46373_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46373.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 417441 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [n:127.0.0.1:46373_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46373.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5c1754fa
[junit4] 2> 417442 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [n:127.0.0.1:46373_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/build/solr-core/test/J1/temp/solr.cloud.api.collections.TestCollectionsAPIViaSolrCloudCluster_CE06C185C04F8F23-001/tempDir-001/node6/.
[junit4] 2> 417454 INFO (zkConnectionManagerCallback-1079-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 417455 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [n:127.0.0.1:46373_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (5)
[junit4] 2> 417455 INFO (TEST-TestCollectionsAPIViaSolrCloudCluster.testCollectionCreateSearchDelete-seed#[CE06C185C04F8F23]) [n:127.0.0.1:46373_solr ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45285/solr ready
[junit4] 2> 417544 INFO (qtp1663875874-4066) [n:127.0.0.1:37627_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=solrCloudCollectionConfig&maxShardsPerNode=1&name=testcollection&nrtReplicas=2&action=CREATE&numShards=2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 417546 INFO (OverseerThreadFactory-1770-thread-1) [ ] o.a.s.c.a.c.CreateCollectionCmd Create collection testcollection
[junit4] 2> 417654 INFO (OverseerStateUpdate-72459470498889740-127.0.0.1:44816_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testcollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43194/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 417657 INFO (OverseerStateUpdate-72459470498889740-127.0.0.1:44816_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"testcollection_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:46373/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 417659 INFO (OverseerStateUpdate-72459470498889740-127.0.0.1:44816_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testcollection_shard2_replica_n4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:44816/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 417661 INFO (OverseerStateUpdate-72459470498889740-127.0.0.1:44816_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"testcollection",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"testcollection_shard2_replica_n6",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:37627/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 417909 INFO (qtp1663875874-4099) [n:127.0.0.1:37627_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node8&name=testcollection_shard2_replica_n6&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 417932 INFO (qtp696035590-4226) [n:127.0.0.1:46373_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node5&name=testcollection_shard1_replica_n2&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 417933 INFO (qtp401764514-4108) [n:127.0.0.1:44816_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node7&name=testcollection_shard2_replica_n4&action=CREATE&numShards=2&shard=shard2&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 417939 INFO (qtp1475404261-4077) [n:127.0.0.1:43194_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=solrCloudCollectionConfig&newCollection=true&collection=testcollection&version=2&replicaType=NRT&coreNodeName=core_node3&name=testcollection_shard1_replica_n1&action=CREATE&numShards=2&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 418063 INFO (zkCallback-1071-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 418063 INFO (zkCallback-1019-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 418063 INFO (zkCallback-1027-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 418063 INFO (zkCallback-1027-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 418063 INFO (zkCallback-1005-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 418066 INFO (zkCallback-1019-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/testcollection/state.json] for collection [testcollection] has occurred - updating... (live nodes size: [5])
[junit4] 2> 418075 INFO (zkCallback-1071-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path
[...truncated too long message...]
ure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null111606620
[copy] Copying 40 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null111606620
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/null111606620
resolve-example:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve-server:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
ivy-availability-check:
[loadresource] Do not set property disallowed.ivy.jars.list as its length is 0.
-ivy-fail-disallowed-ivy-version:
ivy-fail:
ivy-fail:
ivy-configure:
[ivy:configure] :: loading settings :: file = /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/lucene/top-level-ivy-settings.xml
resolve:
jar-checksums:
[mkdir] Created dir: /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1110868827
[copy] Copying 246 files to /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1110868827
[delete] Deleting directory /home/jenkins/jenkins-slave/workspace/Lucene-Solr-BadApples-Tests-7.x/solr/null1110868827
check-working-copy:
[ivy:cachepath] :: resolving dependencies :: org.eclipse.jgit#org.eclipse.jgit-caller;working
[ivy:cachepath] confs: [default]
[ivy:cachepath] found org.eclipse.jgit#org.eclipse.jgit;4.6.0.201612231935-r in public
[ivy:cachepath] found com.jcraft#jsch;0.1.53 in public
[ivy:cachepath] found com.googlecode.javaewah#JavaEWAH;1.1.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpclient;4.3.6 in public
[ivy:cachepath] found org.apache.httpcomponents#httpcore;4.3.3 in public
[ivy:cachepath] found commons-logging#commons-logging;1.1.3 in public
[ivy:cachepath] found commons-codec#commons-codec;1.6 in public
[ivy:cachepath] found org.slf4j#slf4j-api;1.7.2 in public
[ivy:cachepath] :: resolution report :: resolve 52ms :: artifacts dl 1ms
---------------------------------------------------------------------
| | modules || artifacts |
| conf | number| search|dwnlded|evicted|| number|dwnlded|
---------------------------------------------------------------------
| default | 8 | 0 | 0 | 0 || 8 | 0 |
---------------------------------------------------------------------
[wc-checker] Initializing working copy...
[wc-checker] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
[wc-checker] SLF4J: Defaulting to no-operation (NOP) logger implementation
[wc-checker] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
[wc-checker] Checking working copy status...
-jenkins-base:
BUILD SUCCESSFUL
Total time: 128 minutes 14 seconds
Archiving artifacts
WARN: No artifacts found that match the file pattern "**/*.events,heapdumps/**,**/hs_err_pid*". Configuration error?
WARN: java.lang.InterruptedException: no matches found within 10000
Recording test results
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)