You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/01/14 20:58:34 UTC
[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_144) - Build #
21274 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21274/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseConcMarkSweepGC
2 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=7210, name=jetty-launcher-1345-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505) 2) Thread[id=7212, name=jetty-launcher-1345-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=7210, name=jetty-launcher-1345-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
2) Thread[id=7212, name=jetty-launcher-1345-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
at __randomizedtesting.SeedInfo.seed([607345A73B2456EA]:0)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig
Error Message:
expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{ "cluster-preferences":[{"maximize":"freedisk"}], "triggers":{".auto_add_replicas":{ "name":".auto_add_replicas", "event":"nodeLost", "waitFor":30, "actions":[ { "name":"auto_add_replicas_plan", "class":"solr.AutoAddReplicasPlanAction"}, { "name":"execute_plan", "class":"solr.ExecutePlanAction"}], "enabled":true}}, "listeners":{".auto_add_replicas.system":{ "trigger":".auto_add_replicas", "afterAction":[], "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED", "BEFORE_ACTION", "AFTER_ACTION", "IGNORED"], "class":"org.apache.solr.cloud.autoscaling.SystemLogListener", "beforeAction":[]}}, "properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{ "cluster-preferences":[{"maximize":"freedisk"}], "triggers":{".auto_add_replicas":{ "name":".auto_add_replicas", "event":"nodeLost", "waitFor":30, "actions":[ { "name":"auto_add_replicas_plan", "class":"solr.AutoAddReplicasPlanAction"}, { "name":"execute_plan", "class":"solr.ExecutePlanAction"}], "enabled":true}}, "listeners":{".auto_add_replicas.system":{ "trigger":".auto_add_replicas", "afterAction":[], "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED", "BEFORE_ACTION", "AFTER_ACTION", "IGNORED"], "class":"org.apache.solr.cloud.autoscaling.SystemLogListener", "beforeAction":[]}}, "properties":{}}>
Stack Trace:
java.lang.AssertionError: expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
"cluster-preferences":[{"maximize":"freedisk"}],
"triggers":{".auto_add_replicas":{
"name":".auto_add_replicas",
"event":"nodeLost",
"waitFor":30,
"actions":[
{
"name":"auto_add_replicas_plan",
"class":"solr.AutoAddReplicasPlanAction"},
{
"name":"execute_plan",
"class":"solr.ExecutePlanAction"}],
"enabled":true}},
"listeners":{".auto_add_replicas.system":{
"trigger":".auto_add_replicas",
"afterAction":[],
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED",
"BEFORE_ACTION",
"AFTER_ACTION",
"IGNORED"],
"class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
"beforeAction":[]}},
"properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
"cluster-preferences":[{"maximize":"freedisk"}],
"triggers":{".auto_add_replicas":{
"name":".auto_add_replicas",
"event":"nodeLost",
"waitFor":30,
"actions":[
{
"name":"auto_add_replicas_plan",
"class":"solr.AutoAddReplicasPlanAction"},
{
"name":"execute_plan",
"class":"solr.ExecutePlanAction"}],
"enabled":true}},
"listeners":{".auto_add_replicas.system":{
"trigger":".auto_add_replicas",
"afterAction":[],
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED",
"BEFORE_ACTION",
"AFTER_ACTION",
"IGNORED"],
"class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
"beforeAction":[]}},
"properties":{}}>
at __randomizedtesting.SeedInfo.seed([607345A73B2456EA:5FFB460F2C72A60D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig(TestClusterStateProvider.java:214)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12344 lines...]
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
[junit4] 2> 833200 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/init-core-data-001
[junit4] 2> 833201 WARN (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2420 numCloses=2420
[junit4] 2> 833201 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 833202 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 833214 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/tempDir-001
[junit4] 2> 833214 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 833214 INFO (Thread-1960) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 833214 INFO (Thread-1960) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 833215 ERROR (Thread-1960) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 833314 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:45301
[junit4] 2> 833316 INFO (zkConnectionManagerCallback-1348-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 833318 INFO (jetty-launcher-1345-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 833318 INFO (jetty-launcher-1345-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 833319 INFO (jetty-launcher-1345-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19b199e{/solr,null,AVAILABLE}
[junit4] 2> 833319 INFO (jetty-launcher-1345-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7870{/solr,null,AVAILABLE}
[junit4] 2> 833322 INFO (jetty-launcher-1345-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1c41bd6{SSL,[ssl, http/1.1]}{127.0.0.1:44219}
[junit4] 2> 833322 INFO (jetty-launcher-1345-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1410f10{SSL,[ssl, http/1.1]}{127.0.0.1:40941}
[junit4] 2> 833322 INFO (jetty-launcher-1345-thread-1) [ ] o.e.j.s.Server Started @834929ms
[junit4] 2> 833322 INFO (jetty-launcher-1345-thread-2) [ ] o.e.j.s.Server Started @834929ms
[junit4] 2> 833322 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44219}
[junit4] 2> 833322 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40941}
[junit4] 2> 833322 ERROR (jetty-launcher-1345-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 833322 ERROR (jetty-launcher-1345-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 833323 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 833323 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 833323 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 833323 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 833323 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 833323 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 833323 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-14T20:12:10.827Z
[junit4] 2> 833323 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-14T20:12:10.827Z
[junit4] 2> 833327 INFO (zkConnectionManagerCallback-1351-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 833327 INFO (zkConnectionManagerCallback-1352-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 833327 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 833327 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 833336 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45301/solr
[junit4] 2> 833336 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45301/solr
[junit4] 2> 833337 INFO (zkConnectionManagerCallback-1358-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 833337 INFO (zkConnectionManagerCallback-1360-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 833337 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f64d045f0004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 833338 INFO (zkConnectionManagerCallback-1363-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 833338 INFO (zkConnectionManagerCallback-1364-thread-1-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 833368 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 833368 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 833369 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40941_solr
[junit4] 2> 833369 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44219_solr
[junit4] 2> 833369 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.Overseer Overseer (id=99350002448400390-127.0.0.1:40941_solr-n_0000000000) starting
[junit4] 2> 833369 INFO (zkCallback-1362-thread-1-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 833370 INFO (zkCallback-1361-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 833373 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40941_solr
[junit4] 2> 833374 INFO (zkCallback-1361-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 833374 INFO (zkCallback-1362-thread-1-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 833498 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 833502 WARN (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 833506 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=localhost.localdomain,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=jenkins, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 833541 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 833541 WARN (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 833541 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=localhost.localdomain,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=jenkins, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 833562 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 833562 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 833576 INFO (jetty-launcher-1345-thread-1-EventThread) [n:127.0.0.1:44219_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 833576 INFO (jetty-launcher-1345-thread-2-EventThread) [n:127.0.0.1:40941_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 833644 WARN (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 833644 WARN (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 833676 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44219.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
[junit4] 2> 833677 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40941.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
[junit4] 2> 833684 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44219.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
[junit4] 2> 833684 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44219.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
[junit4] 2> 833685 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/tempDir-001/node1/.
[junit4] 2> 833686 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40941.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
[junit4] 2> 833686 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40941.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
[junit4] 2> 833687 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/tempDir-001/node2/.
[junit4] 2> 833725 INFO (zkConnectionManagerCallback-1370-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 833735 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
[junit4] 2> 833784 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noGroups&p=noGroups&t=test&e=1515996731275&s=r5PNXDpWhqFdK6ivqVFRSzQ0JQ8="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:11 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:11 GMT
[junit4] 2> 833785 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
[junit4] 2> 833795 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
[junit4] 2> 839013 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=wrongHost&p=wrongHost&t=test&e=1515996731299&s=Nj/wxDF18s56Os3pCRy1hZaMgNg="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:11 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:11 GMT
[junit4] 2> 839014 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
[junit4] 2> 839030 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
[junit4] 2> 839034 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 839034 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 839034 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515996736535&s=GvjxvsvNTp+JnnsjIQ6/3rwppOw="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:16 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:16 GMT
[junit4] 2> 839034 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
[junit4] 2> 839044 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
[junit4] 2> 839046 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noHosts&p=noHosts&t=test&e=1515996736549&s=p7KCygViyWK2/NVnnwUL1sNFQWc="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:16 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:16 GMT
[junit4] 2> 839047 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
[junit4] 2> 839056 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
[junit4] 2> 839056 INFO (qtp9631235-7160) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 839056 INFO (qtp9631235-7160) [n:127.0.0.1:40941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 839057 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=test&e=1515996736560&s=EKfjYIxpM5wXFVKn8OjSTYKjT6A="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:16 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:16 GMT
[junit4] 2> 839057 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
[junit4] 2> 839065 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
[junit4] 2> 839069 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515996736569&s=rXAXy6lRq9tjZ/cAhNPgXoN36N0="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:16 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:16 GMT
[junit4] 2> 839070 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
[junit4] 2> 839079 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyUnknownRemote
[junit4] 2> 850729 WARN (jetty-launcher-1345-thread-1-SendThread(127.0.0.1:45301)) [n:127.0.0.1:44219_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17044ms for sessionid 0x160f64d045f0005
[junit4] 2> 850729 WARN (jetty-launcher-1345-thread-1-SendThread(127.0.0.1:45301)) [n:127.0.0.1:44219_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17088ms for sessionid 0x160f64d045f0008
[junit4] 2> 850729 WARN (jetty-launcher-1345-thread-2-SendThread(127.0.0.1:45301)) [n:127.0.0.1:40941_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17088ms for sessionid 0x160f64d045f0007
[junit4] 2> 850729 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f64d045f0008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 850730 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f64d045f0005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 850730 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f64d045f0007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 850731 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515996736583&s=dfmuzR/I2V4b/aD/BcZkvv6i/TU="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:16 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:16 GMT
[junit4] 2> 850731 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyUnknownRemote
[junit4] 2> 850743 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
[junit4] 2> 850754 WARN (qtp9631235-7174) [ ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: ???bar???: no such user
[junit4] 2> id: ???bar???: no such user
[junit4] 2>
[junit4] 2> 850756 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=test&e=1515996748247&s=GPRjGGItE1UyIIwdYDtH4BjZtHw="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:28 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:28 GMT
[junit4] 2> 850756 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
[junit4] 2> 850767 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
[junit4] 2> 850788 INFO (qtp9631235-7160) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 850788 INFO (qtp9631235-7160) [n:127.0.0.1:40941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 850788 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostUsersGroup&p=anyHostUsersGroup&t=test&e=1515996748272&s=z9W6+Z/X8M+3wJWuCDoy3HgbeQU="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:28 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:28 GMT
[junit4] 2> 850788 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
[junit4] 2> 850799 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
[junit4] 2> 850801 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1515996748303&s=SbEbEsOm7jevbmKVudIKWskVYVI="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:28 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:28 GMT
[junit4] 2> 850801 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
[junit4] 2> 850814 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
[junit4] 2> 850814 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.s.SolrDispatchFilter Error authenticating
[junit4] 2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:449)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 850815 WARN (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.e.j.s.ServletHandler /solr/admin/collections
[junit4] 2> org.apache.solr.common.SolrException: Error during request authentication,
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:455)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:449)
[junit4] 2> ... 29 more
[junit4] 2> 850816 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
[junit4] 2> 850829 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
[junit4] 2> 850830 INFO (jetty-launcher-1345-thread-1-EventThread) [n:127.0.0.1:44219_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 850830 WARN (zkConnectionManagerCallback-1363-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11c0caf name: ZooKeeperConnection Watcher:127.0.0.1:45301/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
[junit4] 2> 850830 WARN (zkConnectionManagerCallback-1363-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
[junit4] 2> 850835 INFO (zkConnectionManagerCallback-1373-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 851090 INFO (zkConnectionManagerCallback-1376-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 851091 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 851092 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45301/solr ready
[junit4] 2> 851094 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 851095 INFO (OverseerThreadFactory-3228-thread-1-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.CreateCollectionCmd Create collection forwardingCollection
[junit4] 2> 851213 INFO (OverseerStateUpdate-99350002448400390-127.0.0.1:40941_solr-n_0000000000) [n:127.0.0.1:40941_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"forwardingCollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"forwardingCollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:40941/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 851340 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 851340 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 851443 INFO (zkCallback-1362-thread-1-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 851443 INFO (zkCallback-1362-thread-2-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 851943 INFO (jetty-launcher-1345-thread-2-EventThread) [n:127.0.0.1:40941_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 852354 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 852373 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
[junit4] 2> 852499 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 852506 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'forwardingCollection_shard1_replica_n1' using configuration from collection forwardingCollection, trusted=true
[junit4] 2> 852507 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40941.solr.core.forwardingCollection.shard1.replica_n1' (registry 'solr.core.forwardingCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
[junit4] 2> 852507 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 852507 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [[forwardingCollection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/tempDir-001/node2/forwardingCollection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/tempDir-001/node2/./forwardingCollection_shard1_replica_n1/data/]
[junit4] 2> 852510 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=65.8125, floorSegmentMB=1.486328125, forceMergeDeletesPctAllowed=28.468254921113118, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3865847439261596
[junit4] 2> 852539 INFO (zkConnectionManagerCallback-1363-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 852556 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 852556 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 852557 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 852557 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 852558 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=94.5517578125, floorSegmentMB=1.3544921875, forceMergeDeletesPctAllowed=17.983388940845256, segmentsPerTier=22.0, maxCFSSegmentSizeMB=1.009765625, noCFSRatio=1.0
[junit4] 2> 852558 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@663f4a[forwardingCollection_shard1_replica_n1] main]
[junit4] 2> 852559 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 852614 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 852701 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 852701 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 852701 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589600059606958080
[junit4] 2> 852701 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 852701 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 852701 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 852702 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 852702 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 852702 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 852702 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 852702 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 852703 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] Registered new searcher Searcher@663f4a[forwardingCollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:40941/solr/forwardingCollection_shard1_replica_n1/
[junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:40941/solr/forwardingCollection_shard1_replica_n1/ has no replicas
[junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 852706 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:40941/solr/forwardingCollection_shard1_replica_n1/ shard1
[junit4] 2> 852808 INFO (zkCallback-1362-thread-2-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852808 INFO (zkCallback-1362-thread-3-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852857 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 852859 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1519
[junit4] 2> 852860 WARN (httpShardExecutor-3222-thread-1-processing-n:127.0.0.1:40941_solr [https:////127.0.0.1:40941//solr] https:////127.0.0.1:40941//solr) [n:127.0.0.1:40941_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515996748843&s=ymWWTanFXlfJnkYUi/r+HaMKeVA="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:28 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:28 GMT
[junit4] 2> 852868 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 852959 INFO (zkCallback-1362-thread-3-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 852959 INFO (zkCallback-1362-thread-2-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 853102 INFO (OverseerCollectionConfigSetProcessor-99350002448400390-127.0.0.1:40941_solr-n_0000000000) [n:127.0.0.1:40941_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 853868 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=2774
[junit4] 2> 853869 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515996748597&s=SXUJgIGdLeEuFacohlPCTzBHrDY="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:28 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:28 GMT
[junit4] 2> 853869 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: forwardingCollection failOnTimeout:true timeout (sec):100
[junit4] 2> 853869 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: forwardingCollection
[junit4] 2> 853873 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 853874 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515996751377&s=jWsUk51BbMwaRU7HiGenB8gaeGs="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:31 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:31 GMT
[junit4] 2> 853919 INFO (qtp2018146-7159) [n:127.0.0.1:44219_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 853922 INFO (qtp9631235-7162) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 853923 WARN (qtp2018146-7159) [n:127.0.0.1:44219_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515996751425&s=pt3l9WPnA6CsqnggJLac4bZjslQ="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:31 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:31 GMT
[junit4] 2> 853927 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515996751422&s=UXT5V59Toi0WyvJt6tQR2lShGH0="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:31 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:31 GMT
[junit4] 2> 853927 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515996751425&s=pt3l9WPnA6CsqnggJLac4bZjslQ="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:31 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:31 GMT
[junit4] 2> 853927 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testForwarding
[junit4] 2> 853928 INFO (jetty-closer-1346-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1c41bd6{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 853928 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=15275865
[junit4] 2> 853929 INFO (jetty-closer-1346-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1410f10{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 853929 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 853929 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=8601253
[junit4] 2> 853929 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@c82e2c: rootName = solr_44219, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@96e270
[junit4] 2> 853929 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 853929 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@15fb89c: rootName = solr_40941, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1f144e6
[junit4] 2> 853933 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 853933 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 853933 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@190427d: rootName = solr_44219, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@32710e
[junit4] 2> 853933 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1b2dff5: rootName = solr_40941, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@32710e
[junit4] 2> 853936 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 853936 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7f5d3b: rootName = solr_44219, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@dff10
[junit4] 2> 853937 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 853937 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1c4bd73: rootName = solr_40941, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@dff10
[junit4] 2> 853937 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44219_solr
[junit4] 2> 853937 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 853938 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40941_solr
[junit4] 2> 853938 INFO (zkCallback-1361-thread-2-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 853938 INFO (zkCallback-1362-thread-2-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 853938 ERROR (Thread[Thread-1977,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:44219_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 853939 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 853939 INFO (zkCallback-1362-thread-3-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 853939 INFO (coreCloseExecutor-3239-thread-1) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@1e5f6f3
[junit4] 2> 853939 INFO (coreCloseExecutor-3239-thread-1) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.forwardingCollection.shard1.replica_n1, tag=31848179
[junit4] 2> 853939 INFO (coreCloseExecutor-3239-thread-1) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@13896c: rootName = solr_40941, domain = solr.core.forwardingCollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.forwardingCollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@12ebbf3
[junit4] 2> 853952 INFO (jetty-closer-1346-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@19b199e{/solr,null,UNAVAILABLE}
[junit4] 2> 853957 INFO (coreCloseExecutor-3239-thread-1) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.forwardingCollection.shard1.leader, tag=31848179
[junit4] 2> 853958 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.c.Overseer Overseer (id=99350002448400390-127.0.0.1:40941_solr-n_0000000000) closing
[junit4] 2> 853959 INFO (OverseerStateUpdate-99350002448400390-127.0.0.1:40941_solr-n_0000000000) [n:127.0.0.1:40941_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40941_solr
[junit4] 2> 853959 WARN (OverseerAutoScalingTriggerThread-99350002448400390-127.0.0.1:40941_solr-n_0000000000) [n:127.0.0.1:40941_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 853960 ERROR (Thread[Thread-1978,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:40941_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 853960 INFO (jetty-closer-1346-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7870{/solr,null,UNAVAILABLE}
[junit4] 2> 853961 ERROR (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 853961 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45301 45301
[junit4] 2> 854183 INFO (Thread-1960) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45301 45301
[junit4] 2> 854184 WARN (Thread-1960) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
[junit4] 2> 5 /solr/collections/forwardingCollection/state.json
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 4 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
[junit4] 2>
[junit4] 2> 854194 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
[junit4] 2> Jan 14, 2018 8:12:31 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
[junit4] 2> Jan 14, 2018 8:12:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> SEVERE: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
[junit4] 2> 1) Thread[id=7210, name=jetty-launcher-1345-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> 2) Thread[id=7212, name=jetty-launcher-1345-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> at sun.misc.Unsafe.park(Native Method)
[junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
[junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
[junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
[junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
[junit4] 2> Jan 14, 2018 8:12:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
[junit4] 2> INFO: Starting to interrupt leaked threads:
[junit4] 2> 1) Thread[id=7210, name=jetty-launcher-1345-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> 2) Thread[id=7212, name=jetty-launcher-1345-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
[junit4] 2> 874293 ERROR (jetty-launcher-1345-thread-2-EventThread) [n:127.0.0.1:40941_solr ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
[junit4] 2> java.lang.InterruptedException
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
[junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
[junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
[junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
[junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
[junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
[junit4] 2> at org.apa
[...truncated too long message...]
xecutor-3556-thread-1) [n:127.0.0.1:42819_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore [.system_shard1_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@3d8667
[junit4] 2> 1160127 INFO (coreCloseExecutor-3556-thread-1) [n:127.0.0.1:42819_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core..system.shard1.replica_n2, tag=4032103
[junit4] 2> 1160128 INFO (coreCloseExecutor-3556-thread-1) [n:127.0.0.1:42819_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@c08e7f: rootName = solr_42819, domain = solr.core..system.shard1.replica_n2, service url = null, agent id = null] for registry solr.core..system.shard1.replica_n2 / com.codahale.metrics.MetricRegistry@1b3127c
[junit4] 2> 1160131 INFO (jetty-closer-1551-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1160131 INFO (jetty-closer-1551-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1138f7d: rootName = solr_40241, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7404be
[junit4] 2> 1160131 INFO (jetty-closer-1551-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1160132 INFO (jetty-closer-1551-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40241_solr
[junit4] 2> 1160132 INFO (jetty-closer-1551-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@101fa3: rootName = solr_45485, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7404be
[junit4] 2> 1160132 INFO (zkCallback-1576-thread-3-processing-n:127.0.0.1:42819_solr) [n:127.0.0.1:42819_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1160132 INFO (zkCallback-1572-thread-2-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1160133 INFO (zkCallback-1596-thread-1-processing-n:127.0.0.1:45485_solr) [n:127.0.0.1:45485_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1160133 INFO (jetty-closer-1551-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1160134 INFO (jetty-closer-1551-thread-3) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45485_solr
[junit4] 2> 1160135 INFO (jetty-closer-1551-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1160135 INFO (zkCallback-1576-thread-3-processing-n:127.0.0.1:42819_solr) [n:127.0.0.1:42819_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1160135 INFO (zkCallback-1572-thread-2-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1160135 INFO (jetty-closer-1551-thread-3) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1160135 INFO (zkCallback-1596-thread-1-processing-n:127.0.0.1:45485_solr) [n:127.0.0.1:45485_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1160136 INFO (coreCloseExecutor-3557-thread-1) [n:127.0.0.1:40241_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [.system_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@92f7f6
[junit4] 2> 1160137 INFO (coreCloseExecutor-3557-thread-1) [n:127.0.0.1:40241_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core..system.shard1.replica_n1, tag=9631734
[junit4] 2> 1160137 INFO (coreCloseExecutor-3557-thread-1) [n:127.0.0.1:40241_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@104e79f: rootName = solr_40241, domain = solr.core..system.shard1.replica_n1, service url = null, agent id = null] for registry solr.core..system.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@a9da5f
[junit4] 2> 1160137 INFO (jetty-closer-1551-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@cd808{/solr,null,UNAVAILABLE}
[junit4] 2> 1160141 INFO (coreCloseExecutor-3556-thread-1) [n:127.0.0.1:42819_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=4032103
[junit4] 2> 1160145 INFO (jetty-closer-1551-thread-2) [ ] o.a.s.c.Overseer Overseer (id=99350023575765001-127.0.0.1:42819_solr-n_0000000000) closing
[junit4] 2> 1160145 INFO (OverseerStateUpdate-99350023575765001-127.0.0.1:42819_solr-n_0000000000) [n:127.0.0.1:42819_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42819_solr
[junit4] 2> 1160145 INFO (coreCloseExecutor-3557-thread-1) [n:127.0.0.1:40241_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=9631734
[junit4] 2> 1160145 WARN (OverseerAutoScalingTriggerThread-99350023575765001-127.0.0.1:42819_solr-n_0000000000) [n:127.0.0.1:42819_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1160147 INFO (jetty-closer-1551-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1a7059b{/solr,null,UNAVAILABLE}
[junit4] 2> 1161647 INFO (jetty-closer-1551-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1b8b24a{/solr,null,UNAVAILABLE}
[junit4] 2> 1161648 ERROR (SUITE-TestClusterStateProvider-seed#[607345A73B2456EA]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1161648 INFO (SUITE-TestClusterStateProvider-seed#[607345A73B2456EA]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35357 35357
[junit4] 2> 1176670 INFO (Thread-1472) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35357 35357
[junit4] 2> 1176671 WARN (Thread-1472) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 3 /solr/configs/.system
[junit4] 2> 3 /solr/configs/.system/managed-schema
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 20 /solr/collections/.system/state.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 2 /solr/overseer_elect/election/99350023575764999-127.0.0.1:40241_solr-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_607345A73B2456EA-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1858, maxMBSortInHeap=5.144544667996209, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@cd89e5), locale=da, timezone=Pacific/Truk
[junit4] 2> NOTE: Linux 4.13.0-26-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=237171552,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [TestTolerantUpdateProcessorCloud, TestSolrFieldCacheBean, TestFieldCacheSortRandom, DistribJoinFromCollectionTest, ExitableDirectoryReaderTest, UniqFieldsUpdateProcessorFactoryTest, TestSolrConfigHandler, FullSolrCloudDistribCmdsTest, TestExtendedDismaxParser, CdcrReplicationHandlerTest, DistributedFacetSimpleRefinementLongTailTest, TestExclusionRuleCollectionAccess, TestFieldSortValues, TestFunctionQuery, TestCoreDiscovery, MultiThreadedOCPTest, TestDistributedGrouping, TestSolrDeletionPolicy2, DistributedExpandComponentTest, ReplaceNodeTest, TestJsonRequest, TestRestoreCore, TestSolrCloudWithKerberosAlt, TestAuthenticationFramework, AliasIntegrationTest, TestReplicaProperties, TestPerFieldSimilarityWithDefaultOverride, LargeFieldTest, CdcrUpdateLogTest, TestNestedDocsSort, FieldMutatingUpdateProcessorTest, TestTrieFacet, TestCollectionAPI, ShardSplitTest, DocValuesNotIndexedTest, PreAnalyzedUpdateProcessorTest, NodeAddedTriggerTest, TestManagedSchemaThreadSafety, MetricsConfigTest, UnloadDistributedZkTest, TestImpersonationWithHadoopAuth, TestPartialUpdateDeduplication, TestManagedSynonymFilterFactory, SolrCoreCheckLockOnStartupTest, TestMacros, TestCloudJSONFacetJoinDomain, TestBackupRepositoryFactory, BooleanFieldTest, TestDeleteCollectionOnDownNodes, TestLFUCache, CSVRequestHandlerTest, HighlighterTest, IndexBasedSpellCheckerTest, TestReload, TestStressInPlaceUpdates, TestClusterStateProvider]
[junit4] Completed [274/767 (2!)] on J2 in 21.09s, 2 tests, 1 failure <<< FAILURES!
[...truncated 45454 lines...]
[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-9.0.1) - Build #
21278 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21278/
Java: 64bit/jdk-9.0.1 -XX:+UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Could not find collection:collection2
Stack Trace:
java.lang.AssertionError: Could not find collection:collection2
at __randomizedtesting.SeedInfo.seed([DB00226AF17A5DA1:53541DB05F863059]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:913)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:612)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 12415 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> 578733 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[DB00226AF17A5DA1]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/init-core-data-001
[junit4] 2> 578734 WARN (SUITE-FullSolrCloudDistribCmdsTest-seed#[DB00226AF17A5DA1]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=7 numCloses=7
[junit4] 2> 578734 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[DB00226AF17A5DA1]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 578734 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[DB00226AF17A5DA1]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl="https://issues.apache.org/jira/browse/SOLR-5776")
[junit4] 2> 578734 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[DB00226AF17A5DA1]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 578735 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 578735 INFO (Thread-1505) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 578735 INFO (Thread-1505) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 578737 ERROR (Thread-1505) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 578836 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.ZkTestServer start zk server on port:45231
[junit4] 2> 578839 INFO (zkConnectionManagerCallback-1497-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578842 INFO (zkConnectionManagerCallback-1499-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578847 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 578848 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 578849 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 578850 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 578851 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 578852 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 578853 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 578854 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 578855 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 578855 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 578856 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 578857 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 578932 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 578933 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3e16f40b{/,null,AVAILABLE}
[junit4] 2> 578934 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@12ebe179{HTTP/1.1,[http/1.1]}{127.0.0.1:39325}
[junit4] 2> 578934 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.Server Started @580469ms
[junit4] 2> 578934 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/tempDir-001/control/data, hostPort=39325, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/control-001/cores}
[junit4] 2> 578934 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 578934 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 578934 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 578934 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 578934 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T16:02:40.914438Z
[junit4] 2> 578935 INFO (zkConnectionManagerCallback-1501-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578936 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 578936 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/control-001/solr.xml
[junit4] 2> 578937 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 578938 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 578938 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 578940 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45231/solr
[junit4] 2> 578941 INFO (zkConnectionManagerCallback-1505-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578942 INFO (zkConnectionManagerCallback-1507-thread-1-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 578969 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:39325_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 578969 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:39325_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:39325_
[junit4] 2> 578969 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:39325_ ] o.a.s.c.Overseer Overseer (id=99354683684683780-127.0.0.1:39325_-n_0000000000) starting
[junit4] 2> 578972 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:39325_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39325_
[junit4] 2> 578972 INFO (OverseerStateUpdate-99354683684683780-127.0.0.1:39325_-n_0000000000) [n:127.0.0.1:39325_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 578995 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:39325_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 579001 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:39325_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 579001 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:39325_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 579002 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:39325_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/control-001/cores
[junit4] 2> 579012 INFO (zkConnectionManagerCallback-1513-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 579012 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 579013 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45231/solr ready
[junit4] 2> 579014 INFO (qtp594049452-6603) [n:127.0.0.1:39325_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:39325_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 579015 INFO (OverseerThreadFactory-2839-thread-1-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 579120 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 579120 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 579224 INFO (zkCallback-1506-thread-1-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 580155 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 580168 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 580248 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 580254 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 580254 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 580254 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 580254 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 580256 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=876650740, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 580257 WARN (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 580281 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 580281 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 580281 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 580281 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 580282 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@e509d33
[junit4] 2> 580282 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@d81187f[control_collection_shard1_replica_n1] main]
[junit4] 2> 580283 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 580283 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 580283 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 580284 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589674940578136064
[junit4] 2> 580284 INFO (searcherExecutor-2842-thread-1-processing-n:127.0.0.1:39325_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@d81187f[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 580286 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 580287 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 580287 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:39325/control_collection_shard1_replica_n1/
[junit4] 2> 580287 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 580287 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:39325/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 580287 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 580288 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:39325/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 580389 INFO (zkCallback-1506-thread-2-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 580438 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 580440 INFO (qtp594049452-6605) [n:127.0.0.1:39325_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1320
[junit4] 2> 580443 INFO (qtp594049452-6603) [n:127.0.0.1:39325_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 580541 INFO (zkCallback-1506-thread-2-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 581017 INFO (OverseerCollectionConfigSetProcessor-99354683684683780-127.0.0.1:39325_-n_0000000000) [n:127.0.0.1:39325_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 581443 INFO (qtp594049452-6603) [n:127.0.0.1:39325_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:39325_&wt=javabin&version=2} status=0 QTime=2429
[junit4] 2> 581447 INFO (zkConnectionManagerCallback-1517-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581449 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 581450 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45231/solr ready
[junit4] 2> 581450 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 581451 INFO (qtp594049452-6603) [n:127.0.0.1:39325_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 581453 INFO (OverseerThreadFactory-2839-thread-2-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 581454 WARN (OverseerThreadFactory-2839-thread-2-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 581659 INFO (qtp594049452-6603) [n:127.0.0.1:39325_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 581659 INFO (qtp594049452-6603) [n:127.0.0.1:39325_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=207
[junit4] 2> 581766 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-1-001 of type NRT
[junit4] 2> 581766 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 581767 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@23b4fc4f{/,null,AVAILABLE}
[junit4] 2> 581767 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7c878c1{HTTP/1.1,[http/1.1]}{127.0.0.1:36191}
[junit4] 2> 581767 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.Server Started @583303ms
[junit4] 2> 581767 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/tempDir-001/jetty1, hostPort=36191, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-1-001/cores, replicaType=NRT}
[junit4] 2> 581768 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 581768 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 581768 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 581768 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 581768 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T16:02:43.748275Z
[junit4] 2> 581769 INFO (zkConnectionManagerCallback-1519-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581769 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 581769 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-1-001/solr.xml
[junit4] 2> 581771 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 581771 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 581771 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 581773 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45231/solr
[junit4] 2> 581774 INFO (zkConnectionManagerCallback-1523-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581775 INFO (zkConnectionManagerCallback-1525-thread-1-processing-n:127.0.0.1:36191_) [n:127.0.0.1:36191_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 581777 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:36191_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 581778 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:36191_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 581779 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:36191_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:36191_
[junit4] 2> 581780 INFO (zkCallback-1506-thread-2-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 581780 INFO (zkCallback-1516-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 581780 INFO (zkCallback-1524-thread-1-processing-n:127.0.0.1:36191_) [n:127.0.0.1:36191_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 581812 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:36191_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 581819 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:36191_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 581819 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:36191_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 581820 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:36191_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-1-001/cores
[junit4] 2> 581837 INFO (qtp1682519030-6651) [n:127.0.0.1:36191_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:36191_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 581838 INFO (OverseerCollectionConfigSetProcessor-99354683684683780-127.0.0.1:39325_-n_0000000000) [n:127.0.0.1:39325_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 581839 INFO (OverseerThreadFactory-2839-thread-3-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:36191_ for creating new replica
[junit4] 2> 581840 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 581840 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 581942 INFO (zkCallback-1524-thread-1-processing-n:127.0.0.1:36191_) [n:127.0.0.1:36191_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 582852 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 582864 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema [collection1_shard2_replica_n61] Schema name=test
[junit4] 2> 582923 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 582931 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n61' using configuration from collection collection1, trusted=true
[junit4] 2> 582931 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n61' (registry 'solr.core.collection1.shard2.replica_n61') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 582932 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 582932 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [[collection1_shard2_replica_n61] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-1-001/cores/collection1_shard2_replica_n61], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-1-001/cores/collection1_shard2_replica_n61/data/]
[junit4] 2> 582934 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=876650740, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 582936 WARN (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 582988 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 582988 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 582989 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 582989 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 582991 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@35c73fd6
[junit4] 2> 582991 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.SolrIndexSearcher Opening [Searcher@b32e7e1[collection1_shard2_replica_n61] main]
[junit4] 2> 582992 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 582992 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 582992 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 582993 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589674943418728448
[junit4] 2> 582993 INFO (searcherExecutor-2853-thread-1-processing-n:127.0.0.1:36191_ x:collection1_shard2_replica_n61 s:shard2 c:collection1) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SolrCore [collection1_shard2_replica_n61] Registered new searcher Searcher@b32e7e1[collection1_shard2_replica_n61] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 582996 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 582996 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 582996 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:36191/collection1_shard2_replica_n61/
[junit4] 2> 582996 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 582996 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.SyncStrategy http://127.0.0.1:36191/collection1_shard2_replica_n61/ has no replicas
[junit4] 2> 582996 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 582997 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:36191/collection1_shard2_replica_n61/ shard2
[junit4] 2> 583098 INFO (zkCallback-1524-thread-1-processing-n:127.0.0.1:36191_) [n:127.0.0.1:36191_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 583148 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 583150 INFO (qtp1682519030-6653) [n:127.0.0.1:36191_ c:collection1 s:shard2 x:collection1_shard2_replica_n61] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n61&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1310
[junit4] 2> 583153 INFO (qtp1682519030-6651) [n:127.0.0.1:36191_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:36191_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=1315
[junit4] 2> 583230 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-2-001 of type NRT
[junit4] 2> 583230 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 583231 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@235647b6{/,null,AVAILABLE}
[junit4] 2> 583231 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@65839527{HTTP/1.1,[http/1.1]}{127.0.0.1:46241}
[junit4] 2> 583231 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.Server Started @584767ms
[junit4] 2> 583231 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/tempDir-001/jetty2, hostPort=46241, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-2-001/cores, replicaType=NRT}
[junit4] 2> 583231 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 583231 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 583231 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 583232 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 583232 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T16:02:45.212015Z
[junit4] 2> 583232 INFO (zkConnectionManagerCallback-1529-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583233 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 583233 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-2-001/solr.xml
[junit4] 2> 583234 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 583234 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 583235 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 583236 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45231/solr
[junit4] 2> 583237 INFO (zkConnectionManagerCallback-1533-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583239 INFO (zkConnectionManagerCallback-1535-thread-1-processing-n:127.0.0.1:46241_) [n:127.0.0.1:46241_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 583240 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:46241_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 583241 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:46241_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 583242 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:46241_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46241_
[junit4] 2> 583242 INFO (zkCallback-1506-thread-2-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 583242 INFO (zkCallback-1524-thread-1-processing-n:127.0.0.1:36191_) [n:127.0.0.1:36191_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 583242 INFO (zkCallback-1516-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 583242 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:46241_) [n:127.0.0.1:46241_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 583269 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:46241_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 583276 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:46241_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 583276 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:46241_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 583277 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:46241_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-2-001/cores
[junit4] 2> 583297 INFO (qtp1682519030-6652) [n:127.0.0.1:36191_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:46241_&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 583298 INFO (OverseerCollectionConfigSetProcessor-99354683684683780-127.0.0.1:39325_-n_0000000000) [n:127.0.0.1:39325_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 583299 INFO (OverseerThreadFactory-2839-thread-4-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:46241_ for creating new replica
[junit4] 2> 583300 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n63&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 583300 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 583402 INFO (zkCallback-1524-thread-1-processing-n:127.0.0.1:36191_) [n:127.0.0.1:36191_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 583402 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:46241_) [n:127.0.0.1:46241_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 584308 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 584330 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema [collection1_shard3_replica_n63] Schema name=test
[junit4] 2> 584386 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 584393 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard3_replica_n63' using configuration from collection collection1, trusted=true
[junit4] 2> 584393 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard3.replica_n63' (registry 'solr.core.collection1.shard3.replica_n63') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 584393 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 584393 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore [[collection1_shard3_replica_n63] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-2-001/cores/collection1_shard3_replica_n63], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-2-001/cores/collection1_shard3_replica_n63/data/]
[junit4] 2> 584394 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=876650740, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 584396 WARN (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 584432 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 584432 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 584433 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 584433 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 584434 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@33249c05
[junit4] 2> 584434 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.SolrIndexSearcher Opening [Searcher@59fcf420[collection1_shard3_replica_n63] main]
[junit4] 2> 584435 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 584435 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 584435 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 584436 INFO (searcherExecutor-2864-thread-1-processing-n:127.0.0.1:46241_ x:collection1_shard3_replica_n63 s:shard3 c:collection1) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SolrCore [collection1_shard3_replica_n63] Registered new searcher Searcher@59fcf420[collection1_shard3_replica_n63] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 584436 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589674944931823616
[junit4] 2> 584439 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 584439 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 584439 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:46241/collection1_shard3_replica_n63/
[junit4] 2> 584439 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 584439 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.SyncStrategy http://127.0.0.1:46241/collection1_shard3_replica_n63/ has no replicas
[junit4] 2> 584439 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 584440 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:46241/collection1_shard3_replica_n63/ shard3
[junit4] 2> 584540 INFO (zkCallback-1524-thread-1-processing-n:127.0.0.1:36191_) [n:127.0.0.1:36191_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 584541 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:46241_) [n:127.0.0.1:46241_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 584590 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 584593 INFO (qtp1117650808-6683) [n:127.0.0.1:46241_ c:collection1 s:shard3 x:collection1_shard3_replica_n63] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n63&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1292
[junit4] 2> 584595 INFO (qtp1682519030-6652) [n:127.0.0.1:36191_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:46241_&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2} status=0 QTime=1298
[junit4] 2> 584672 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-3-001 of type NRT
[junit4] 2> 584673 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 584673 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5fac0aca{/,null,AVAILABLE}
[junit4] 2> 584674 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@34026a6c{HTTP/1.1,[http/1.1]}{127.0.0.1:43793}
[junit4] 2> 584674 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.Server Started @586209ms
[junit4] 2> 584674 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/tempDir-001/jetty3, hostPort=43793, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-3-001/cores, replicaType=NRT}
[junit4] 2> 584674 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 584674 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 584674 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 584674 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 584674 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T16:02:46.654524Z
[junit4] 2> 584675 INFO (zkConnectionManagerCallback-1539-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584676 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 584676 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-3-001/solr.xml
[junit4] 2> 584678 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 584678 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 584678 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 584679 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45231/solr
[junit4] 2> 584680 INFO (zkConnectionManagerCallback-1543-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584681 INFO (zkConnectionManagerCallback-1545-thread-1-processing-n:127.0.0.1:43793_) [n:127.0.0.1:43793_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584683 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:43793_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 584684 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:43793_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 584685 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:43793_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43793_
[junit4] 2> 584685 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:46241_) [n:127.0.0.1:46241_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 584685 INFO (zkCallback-1516-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 584685 INFO (zkCallback-1506-thread-2-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 584686 INFO (zkCallback-1524-thread-1-processing-n:127.0.0.1:36191_) [n:127.0.0.1:36191_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 584686 INFO (zkCallback-1544-thread-1-processing-n:127.0.0.1:43793_) [n:127.0.0.1:43793_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 584707 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:43793_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 584714 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:43793_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 584714 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:43793_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 584715 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:43793_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-3-001/cores
[junit4] 2> 584740 INFO (qtp1682519030-6647) [n:127.0.0.1:36191_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:43793_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 584742 INFO (OverseerCollectionConfigSetProcessor-99354683684683780-127.0.0.1:39325_-n_0000000000) [n:127.0.0.1:39325_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 584742 INFO (OverseerThreadFactory-2839-thread-5-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:43793_ for creating new replica
[junit4] 2> 584743 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n65&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 584743 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 584846 INFO (zkCallback-1524-thread-1-processing-n:127.0.0.1:36191_) [n:127.0.0.1:36191_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 584846 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:46241_) [n:127.0.0.1:46241_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 584846 INFO (zkCallback-1544-thread-1-processing-n:127.0.0.1:43793_) [n:127.0.0.1:43793_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 585783 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 585795 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.IndexSchema [collection1_shard1_replica_n65] Schema name=test
[junit4] 2> 585857 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 585863 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n65' using configuration from collection collection1, trusted=true
[junit4] 2> 585863 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n65' (registry 'solr.core.collection1.shard1.replica_n65') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 585863 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 585863 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrCore [[collection1_shard1_replica_n65] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-3-001/cores/collection1_shard1_replica_n65], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-3-001/cores/collection1_shard1_replica_n65/data/]
[junit4] 2> 585864 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=876650740, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 585866 WARN (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 585888 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 585888 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 585888 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 585888 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 585889 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@2913b67b
[junit4] 2> 585889 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.SolrIndexSearcher Opening [Searcher@25be31ae[collection1_shard1_replica_n65] main]
[junit4] 2> 585890 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 585890 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 585890 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 585891 INFO (searcherExecutor-2875-thread-1-processing-n:127.0.0.1:43793_ x:collection1_shard1_replica_n65 s:shard1 c:collection1) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SolrCore [collection1_shard1_replica_n65] Registered new searcher Searcher@25be31ae[collection1_shard1_replica_n65] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 585891 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589674946457501696
[junit4] 2> 585893 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 585893 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 585893 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43793/collection1_shard1_replica_n65/
[junit4] 2> 585894 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 585894 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.SyncStrategy http://127.0.0.1:43793/collection1_shard1_replica_n65/ has no replicas
[junit4] 2> 585894 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 585895 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43793/collection1_shard1_replica_n65/ shard1
[junit4] 2> 585996 INFO (zkCallback-1544-thread-1-processing-n:127.0.0.1:43793_) [n:127.0.0.1:43793_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 585996 INFO (zkCallback-1524-thread-1-processing-n:127.0.0.1:36191_) [n:127.0.0.1:36191_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 585996 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:46241_) [n:127.0.0.1:46241_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 586045 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 586047 INFO (qtp163357144-6714) [n:127.0.0.1:43793_ c:collection1 s:shard1 x:collection1_shard1_replica_n65] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n65&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1304
[junit4] 2> 586050 INFO (qtp1682519030-6647) [n:127.0.0.1:36191_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:43793_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=1310
[junit4] 2> 586128 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-4-001 of type NRT
[junit4] 2> 586129 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 586130 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1774ab2{/,null,AVAILABLE}
[junit4] 2> 586130 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5a7e9b2e{HTTP/1.1,[http/1.1]}{127.0.0.1:33445}
[junit4] 2> 586130 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.e.j.s.Server Started @587666ms
[junit4] 2> 586130 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/, solrconfig=solrconfig.xml, solr.data.dir=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/tempDir-001/jetty4, hostPort=33445, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-4-001/cores, replicaType=NRT}
[junit4] 2> 586131 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 586131 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 586131 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 586131 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 586131 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T16:02:48.111296Z
[junit4] 2> 586132 INFO (zkConnectionManagerCallback-1549-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 586132 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 586132 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-4-001/solr.xml
[junit4] 2> 586134 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 586134 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 586134 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 586136 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45231/solr
[junit4] 2> 586137 INFO (zkConnectionManagerCallback-1553-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 586138 INFO (zkConnectionManagerCallback-1555-thread-1-processing-n:127.0.0.1:33445_) [n:127.0.0.1:33445_ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 586140 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:33445_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 586140 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:33445_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 586141 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:33445_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33445_
[junit4] 2> 586142 INFO (zkCallback-1534-thread-1-processing-n:127.0.0.1:46241_) [n:127.0.0.1:46241_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 586142 INFO (zkCallback-1516-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 586142 INFO (zkCallback-1544-thread-1-processing-n:127.0.0.1:43793_) [n:127.0.0.1:43793_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 586142 INFO (zkCallback-1524-thread-1-processing-n:127.0.0.1:36191_) [n:127.0.0.1:36191_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 586142 INFO (zkCallback-1554-thread-1-processing-n:127.0.0.1:33445_) [n:127.0.0.1:33445_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 586142 INFO (zkCallback-1506-thread-2-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 586165 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:33445_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 586185 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:33445_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 586185 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:33445_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6702aeb3
[junit4] 2> 586186 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[DB00226AF17A5DA1]) [n:127.0.0.1:33445_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.FullSolrCloudDistribCmdsTest_DB00226AF17A5DA1-001/shard-4-001/cores
[junit4] 2> 586217 INFO (qtp1972513994-6745) [n:127.0.0.1:33445_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:33445_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 586219 INFO (OverseerCollectionConfigSetProcessor-99354683684683780-127.0.0.1:39325_-n_0000000000) [n:127.0.0.1:39325_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 586219 INFO (OverseerThreadFactory-2839-thread-5-processing-n:127.0.0.1:39325_) [n:127.0.0.1:39325_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:33445_ for creating new replica
[junit4] 2> 586220 INFO (qtp1972513994-6747) [n:127.0.0.1:33445_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n67&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 586220 INFO (qtp1972513994-6747) [n:127.0.0.1:33445_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 586323 INFO (zkCallback-
[...truncated too long message...]
ing), 7grand1_s=Lucene50(blocksize=128), 15parent_f2_s=PostingsFormat(name=Direct), 9grand7_s=PostingsFormat(name=Asserting), 25parent_f2_s=Lucene50(blocksize=128), 8grand0_s=PostingsFormat(name=Asserting), 9grand3_s=PostingsFormat(name=Asserting), 6parent_f2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 35parent_f2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 8grand4_s=PostingsFormat(name=Asserting), 0parent_f2_s=PostingsFormat(name=Direct), 3grand1_s=Lucene50(blocksize=128), 19parent_f1_s=Lucene50(blocksize=128), 4parent_f1_s=Lucene50(blocksize=128), 11grand10_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 5child3_s=PostingsFormat(name=Asserting), 9parent_f2_s=Lucene50(blocksize=128), a_t=Lucene50(blocksize=128), 6child2_s=Lucene50(blocksize=128), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), 5grand3_s=Lucene50(blocksize=128), 1parent_f1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 13parent_f1_s=Lucene50(blocksize=128), 23parent_f1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 6child1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), _root_=PostingsFormat(name=Asserting), 5child2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 29parent_f1_s=PostingsFormat(name=Direct), 33parent_f1_s=PostingsFormat(name=Asserting), 39parent_f1_s=Lucene50(blocksize=128), 3child0_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 4child3_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 12parent_f1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 22parent_f1_s=PostingsFormat(name=Asserting), 1grand0_s=PostingsFormat(name=Direct), 11grand2_s=Lucene50(blocksize=128), 11grand6_s=Lucene50(blocksize=128), 10grand7_s=PostingsFormat(name=Asserting), 7parent_f1_s=PostingsFormat(name=Direct), 2child1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 32parent_f1_s=PostingsFormat(name=Direct), 3parent_f2_s=PostingsFormat(name=Asserting), 10grand3_s=PostingsFormat(name=Asserting), 6grand3_s=PostingsFormat(name=Direct), 31parent_f2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 5grand2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 10grand9_s=Lucene50(blocksize=128), 11parent_f2_s=PostingsFormat(name=Direct), 21parent_f2_s=Lucene50(blocksize=128), 4grand1_s=PostingsFormat(name=Direct), 7grand4_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 3grand0_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 9grand6_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 8grand1_s=PostingsFormat(name=Direct), 9grand2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 8grand5_s=PostingsFormat(name=Direct), 9parent_f1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 30parent_f1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 14parent_f2_s=Lucene50(blocksize=128), 20parent_f1_s=Lucene50(blocksize=128), 1parent_f2_s=Lucene50(blocksize=128), 6child3_s=PostingsFormat(name=Direct), 7grand0_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), text_t=PostingsFormat(name=Asserting), 24parent_f2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 34parent_f2_s=PostingsFormat(name=Asserting), 13parent_f2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), type_s=PostingsFormat(name=Asserting), 23parent_f2_s=PostingsFormat(name=Asserting), 5child1_s=Lucene50(blocksize=128), 10parent_f1_s=PostingsFormat(name=Direct), 33parent_f2_s=PostingsFormat(name=Direct), 11grand1_s=PostingsFormat(name=Direct), 22parent_f2_s=PostingsFormat(name=Direct), 10grand0_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 4child0_s=Lucene50(blocksize=128), 11grand5_s=PostingsFormat(name=Direct), 12parent_f2_s=PostingsFormat(name=Asserting), 10grand8_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 10grand4_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 32parent_f2_s=Lucene50(blocksize=128), 31parent_f1_s=Lucene50(blocksize=128), 5grand1_s=PostingsFormat(name=Asserting), 6grand4_s=Lucene50(blocksize=128), 11parent_f1_s=PostingsFormat(name=Asserting), 21parent_f1_s=PostingsFormat(name=Direct), 4grand2_s=Lucene50(blocksize=128), 7grand3_s=PostingsFormat(name=Asserting), 11grand9_s=PostingsFormat(name=Direct), 8grand2_s=Lucene50(blocksize=128), 9grand5_s=Lucene50(blocksize=128), 8grand6_s=Lucene50(blocksize=128), 9grand1_s=Lucene50(blocksize=128), 37parent_f1_s=PostingsFormat(name=Asserting), 14parent_f1_s=PostingsFormat(name=Direct), 20parent_f2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 34parent_f1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 24parent_f1_s=Lucene50(blocksize=128), 6child4_s=PostingsFormat(name=Asserting), 30parent_f2_s=PostingsFormat(name=Asserting), 5parent_f1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 6grand0_s=Lucene50(blocksize=128), 10parent_f2_s=Lucene50(blocksize=128), 5child0_s=PostingsFormat(name=Direct), 8parent_f1_s=Lucene50(blocksize=128), 27parent_f2_s=PostingsFormat(name=Asserting), 2parent_f2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 11grand0_s=PostingsFormat(name=Asserting), 4child1_s=PostingsFormat(name=Direct), 2grand1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 11grand4_s=PostingsFormat(name=Asserting), 17parent_f2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 3child2_s=PostingsFormat(name=Direct), 16parent_f2_s=Lucene50(blocksize=128), 10grand5_s=Lucene50(blocksize=128), 26parent_f2_s=PostingsFormat(name=Direct), 1child0_s=PostingsFormat(name=Direct), 36parent_f2_s=Lucene50(blocksize=128), 10grand1_s=Lucene50(blocksize=128), 7grand6_s=PostingsFormat(name=Direct), 18parent_f1_s=PostingsFormat(name=Direct), 28parent_f1_s=PostingsFormat(name=Asserting), 5grand0_s=PostingsFormat(name=Direct), 6grand5_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 7grand2_s=PostingsFormat(name=Direct), 15parent_f1_s=PostingsFormat(name=Asserting), 4grand3_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 9grand8_s=PostingsFormat(name=Direct), 11grand8_s=PostingsFormat(name=Asserting), 8grand3_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 6parent_f1_s=PostingsFormat(name=Asserting), 25parent_f1_s=PostingsFormat(name=Direct), 9grand4_s=PostingsFormat(name=Direct), 8grand7_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 0parent_f1_s=Lucene50(blocksize=128), 3grand2_s=PostingsFormat(name=Direct), 35parent_f1_s=Lucene50(blocksize=128), 9grand0_s=PostingsFormat(name=Direct), 19parent_f2_s=PostingsFormat(name=Direct), 4parent_f2_s=PostingsFormat(name=Direct), 37parent_f2_s=PostingsFormat(name=Direct), 5child4_s=PostingsFormat(name=Direct), 6child5_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 38parent_f1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 5grand4_s=PostingsFormat(name=Direct), 6grand1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 5parent_f2_s=Lucene50(blocksize=128), 6child0_s=PostingsFormat(name=Asserting), 16parent_f1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 8parent_f2_s=PostingsFormat(name=Direct), 39parent_f2_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 29parent_f2_s=PostingsFormat(name=Asserting), 2parent_f1_s=PostingsFormat(name=Asserting), 4child2_s=PostingsFormat(name=Asserting), rnd_b=PostingsFormat(name=LuceneVarGapDocFreqInterval), 11grand3_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 27parent_f1_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 17parent_f1_s=PostingsFormat(name=Asserting), 2grand0_s=PostingsFormat(name=Asserting), 11grand7_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), 3child1_s=Lucene50(blocksize=128), 10grand6_s=PostingsFormat(name=Direct), 7parent_f2_s=PostingsFormat(name=Asserting), 26parent_f1_s=PostingsFormat(name=Asserting), 10grand2_s=PostingsFormat(name=Direct), 2child0_s=Lucene50(blocksize=128), 3parent_f1_s=PostingsFormat(name=Direct), 36parent_f1_s=PostingsFormat(name=Direct)}, docValues:{_version_=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=520, maxMBSortInHeap=6.754891029703482, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@a0cfe5a), locale=en-KY, timezone=Asia/Amman
[junit4] 2> NOTE: Linux 4.13.0-26-generic amd64/Oracle Corporation 9.0.1 (64-bit)/cpus=8,threads=1,free=140872056,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [TestFiltering, FieldAnalysisRequestHandlerTest, ClusterStateTest, TestClusterStateMutator, SolrSlf4jReporterTest, BlockJoinFacetRandomTest, RankQueryTest, MultiThreadedOCPTest, TestGraphTermsQParserPlugin, ReplicaListTransformerTest, TestBlendedInfixSuggestions, ImplicitSnitchTest, DefaultValueUpdateProcessorTest, AlternateDirectoryTest, SuggesterWFSTTest, AsyncCallRequestStatusResponseTest, TestDocTermOrds, SolrGraphiteReporterTest, IndexBasedSpellCheckerTest, CdcrBidirectionalTest, TlogReplayBufferedWhileIndexingTest, TestBinaryResponseWriter, NumberUtilsTest, AutoScalingHandlerTest, PeerSyncReplicationTest, SolrCLIZkUtilsTest, TestReloadAndDeleteDocs, TestManagedResource, DistributedSpellCheckComponentTest, TestRTimerTree, EchoParamsTest, TestSimDistributedQueue, BufferStoreTest, DistribJoinFromCollectionTest, TestCollapseQParserPlugin, OverseerRolesTest, HdfsAutoAddReplicasIntegrationTest, TestSortByMinMaxFunction, TestCloudJSONFacetJoinDomain, TestSearchPerf, SortByFunctionTest, TestCoreContainer, DistributedExpandComponentTest, ChaosMonkeyNothingIsSafeTest, ConfigureRecoveryStrategyTest, BadComponentTest, TestFieldCacheWithThreads, TestExactStatsCache, HdfsUnloadDistributedZkTest, SolrTestCaseJ4Test, TestElisionMultitermQuery, InfoHandlerTest, TestManagedSynonymFilterFactory, UniqFieldsUpdateProcessorFactoryTest, IndexSchemaTest, DeleteNodeTest, TestNodeAddedTrigger, TestLockTree, TestGroupingSearch, TestBulkSchemaAPI, TestDelegationWithHadoopAuth, TestBackupRepositoryFactory, TestSolrConfigHandlerCloud, TestExecutePlanAction, TestCustomStream, TestExpandComponent, TestRealTimeGet, FieldMutatingUpdateProcessorTest, DistributedFacetPivotWhiteBoxTest, V2ApiIntegrationTest, DistributedFacetPivotLongTailTest, FullSolrCloudDistribCmdsTest]
[junit4] Completed [210/768 (1!)] on J2 in 62.54s, 1 test, 1 failure <<< FAILURES!
[...truncated 42605 lines...]
[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_144) - Build #
21277 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21277/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseParallelGC
2 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestDistribStateManager.testGetSetRemoveData
Error Message:
Node watch should have fired!
Stack Trace:
java.lang.AssertionError: Node watch should have fired!
at __randomizedtesting.SeedInfo.seed([BBF0E0A8DF27CA36:9D600FFA8BA90C3C]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.autoscaling.sim.TestDistribStateManager.testGetSetRemoveData(TestDistribStateManager.java:256)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig
Error Message:
expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{ "cluster-preferences":[{"maximize":"freedisk"}], "triggers":{".auto_add_replicas":{ "name":".auto_add_replicas", "event":"nodeLost", "waitFor":30, "actions":[ { "name":"auto_add_replicas_plan", "class":"solr.AutoAddReplicasPlanAction"}, { "name":"execute_plan", "class":"solr.ExecutePlanAction"}], "enabled":true}}, "listeners":{".auto_add_replicas.system":{ "trigger":".auto_add_replicas", "afterAction":[], "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED", "BEFORE_ACTION", "AFTER_ACTION", "IGNORED"], "class":"org.apache.solr.cloud.autoscaling.SystemLogListener", "beforeAction":[]}}, "properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{ "cluster-preferences":[{"maximize":"freedisk"}], "triggers":{".auto_add_replicas":{ "name":".auto_add_replicas", "event":"nodeLost", "waitFor":30, "actions":[ { "name":"auto_add_replicas_plan", "class":"solr.AutoAddReplicasPlanAction"}, { "name":"execute_plan", "class":"solr.ExecutePlanAction"}], "enabled":true}}, "listeners":{".auto_add_replicas.system":{ "trigger":".auto_add_replicas", "afterAction":[], "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED", "BEFORE_ACTION", "AFTER_ACTION", "IGNORED"], "class":"org.apache.solr.cloud.autoscaling.SystemLogListener", "beforeAction":[]}}, "properties":{}}>
Stack Trace:
java.lang.AssertionError: expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
"cluster-preferences":[{"maximize":"freedisk"}],
"triggers":{".auto_add_replicas":{
"name":".auto_add_replicas",
"event":"nodeLost",
"waitFor":30,
"actions":[
{
"name":"auto_add_replicas_plan",
"class":"solr.AutoAddReplicasPlanAction"},
{
"name":"execute_plan",
"class":"solr.ExecutePlanAction"}],
"enabled":true}},
"listeners":{".auto_add_replicas.system":{
"trigger":".auto_add_replicas",
"afterAction":[],
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED",
"BEFORE_ACTION",
"AFTER_ACTION",
"IGNORED"],
"class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
"beforeAction":[]}},
"properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
"cluster-preferences":[{"maximize":"freedisk"}],
"triggers":{".auto_add_replicas":{
"name":".auto_add_replicas",
"event":"nodeLost",
"waitFor":30,
"actions":[
{
"name":"auto_add_replicas_plan",
"class":"solr.AutoAddReplicasPlanAction"},
{
"name":"execute_plan",
"class":"solr.ExecutePlanAction"}],
"enabled":true}},
"listeners":{".auto_add_replicas.system":{
"trigger":".auto_add_replicas",
"afterAction":[],
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED",
"BEFORE_ACTION",
"AFTER_ACTION",
"IGNORED"],
"class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
"beforeAction":[]}},
"properties":{}}>
at __randomizedtesting.SeedInfo.seed([BBF0E0A8DF27CA36:8478E300C8713AD1]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig(TestClusterStateProvider.java:214)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12117 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestDistribStateManager
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestDistribStateManager_BBF0E0A8DF27CA36-001/init-core-data-001
[junit4] 2> 557407 WARN (SUITE-TestDistribStateManager-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=3 numCloses=3
[junit4] 2> 557408 INFO (SUITE-TestDistribStateManager-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 557409 INFO (SUITE-TestDistribStateManager-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 557409 INFO (SUITE-TestDistribStateManager-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 557410 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testCreateMode
[junit4] 2> 557410 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 557410 INFO (Thread-705) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 557410 INFO (Thread-705) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 557411 ERROR (Thread-705) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 557511 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer start zk server on port:46653
[junit4] 2> 557512 INFO (zkConnectionManagerCallback-854-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 557512 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 557519 INFO (zkConnectionManagerCallback-856-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 557519 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 557520 ERROR (TEST-TestDistribStateManager.testCreateMode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 557520 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46653 46653
[junit4] 2> 562551 INFO (Thread-705) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46653 46653
[junit4] 2> 562551 INFO (TEST-TestDistribStateManager.testCreateMode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testCreateMode
[junit4] 2> 562553 INFO (TEST-TestDistribStateManager.testListData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testListData
[junit4] 2> 562553 INFO (TEST-TestDistribStateManager.testListData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 562553 INFO (Thread-706) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 562553 INFO (Thread-706) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 562556 ERROR (Thread-706) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 562653 INFO (TEST-TestDistribStateManager.testListData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer start zk server on port:35637
[junit4] 2> 562655 INFO (zkConnectionManagerCallback-858-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 562655 INFO (TEST-TestDistribStateManager.testListData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 562658 ERROR (TEST-TestDistribStateManager.testListData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 562658 INFO (TEST-TestDistribStateManager.testListData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35637 35637
[junit4] 2> 568899 INFO (Thread-706) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35637 35637
[junit4] 2> 568899 INFO (TEST-TestDistribStateManager.testListData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testListData
[junit4] 2> 568909 INFO (TEST-TestDistribStateManager.testMulti-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testMulti
[junit4] 2> 568910 INFO (TEST-TestDistribStateManager.testMulti-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 568910 INFO (Thread-707) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 568910 INFO (Thread-707) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 568911 ERROR (Thread-707) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 569010 INFO (TEST-TestDistribStateManager.testMulti-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer start zk server on port:43005
[junit4] 2> 569011 INFO (zkConnectionManagerCallback-860-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 569012 INFO (TEST-TestDistribStateManager.testMulti-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 569012 ERROR (TEST-TestDistribStateManager.testMulti-seed#[BBF0E0A8DF27CA36]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 569012 INFO (TEST-TestDistribStateManager.testMulti-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43005 43005
[junit4] 2> 574508 INFO (Thread-707) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43005 43005
[junit4] 2> 574509 INFO (TEST-TestDistribStateManager.testMulti-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testMulti
[junit4] 2> 574513 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testGetSetRemoveData
[junit4] 2> 574513 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 574515 INFO (Thread-708) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 574515 INFO (Thread-708) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 574533 ERROR (Thread-708) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 574614 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer start zk server on port:35877
[junit4] 2> 574645 INFO (zkConnectionManagerCallback-862-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 574646 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 574661 INFO (zkConnectionManagerCallback-864-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 574661 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 579667 ERROR (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 579667 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35877 35877
[junit4] 2> 584691 INFO (Thread-708) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35877 35877
[junit4] 2> 584691 INFO (TEST-TestDistribStateManager.testGetSetRemoveData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testGetSetRemoveData
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestDistribStateManager -Dtests.method=testGetSetRemoveData -Dtests.seed=BBF0E0A8DF27CA36 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=mt -Dtests.timezone=America/Juneau -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 10.2s J1 | TestDistribStateManager.testGetSetRemoveData <<<
[junit4] > Throwable #1: java.lang.AssertionError: Node watch should have fired!
[junit4] > at __randomizedtesting.SeedInfo.seed([BBF0E0A8DF27CA36:9D600FFA8BA90C3C]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestDistribStateManager.testGetSetRemoveData(TestDistribStateManager.java:256)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 584693 INFO (TEST-TestDistribStateManager.testHasData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testHasData
[junit4] 2> 584694 INFO (TEST-TestDistribStateManager.testHasData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 584694 INFO (Thread-709) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 584694 INFO (Thread-709) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 584697 ERROR (Thread-709) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 584794 INFO (TEST-TestDistribStateManager.testHasData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer start zk server on port:45091
[junit4] 2> 584796 INFO (zkConnectionManagerCallback-866-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 584796 INFO (TEST-TestDistribStateManager.testHasData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.a.s.TestDistribStateManager Using org.apache.solr.client.solrj.impl.ZkDistribStateManager
[junit4] 2> 584798 ERROR (TEST-TestDistribStateManager.testHasData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 584798 INFO (TEST-TestDistribStateManager.testHasData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45091 45091
[junit4] 2> 589909 INFO (Thread-709) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45091 45091
[junit4] 2> 589910 INFO (TEST-TestDistribStateManager.testHasData-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testHasData
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.autoscaling.sim.TestDistribStateManager_BBF0E0A8DF27CA36-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=922, maxMBSortInHeap=6.554842304891407, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@ba7948), locale=mt, timezone=America/Juneau
[junit4] 2> NOTE: Linux 4.13.0-26-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=194593792,total=331874304
[junit4] 2> NOTE: All tests run in this JVM: [TestDocumentBuilder, TestFieldResource, TestFilteredDocIdSet, ZkSolrClientTest, DebugComponentTest, TestFaceting, SampleTest, TestPseudoReturnFields, DistributedQueryComponentOptimizationTest, TriggerIntegrationTest, OverseerTaskQueueTest, SuggesterTSTTest, DeleteShardTest, TestSolrCloudWithDelegationTokens, TestSurroundQueryParser, BlockDirectoryTest, ChaosMonkeySafeLeaderTest, TestScoreJoinQPNoScore, TestObjectReleaseTracker, TestPivotHelperCode, TestJavabinTupleStreamParser, TestFastWriter, PathHierarchyTokenizerFactoryTest, ExplicitHLLTest, TestConfigSetImmutable, TestReRankQParserPlugin, TestLeaderElectionWithEmptyReplica, SynonymTokenizerTest, DocValuesMultiTest, TestSchemaNameResource, DocumentAnalysisRequestHandlerTest, AutoAddReplicasPlanActionTest, TestSearcherReuse, CollectionTooManyReplicasTest, QueryEqualityTest, DOMUtilTest, CdcrReplicationDistributedZkTest, TestDFISimilarityFactory, CircularListTest, TestDistributedStatsComponentCardinality, TestQuerySenderNoQuery, TestImplicitCoreProperties, TestLocalFSCloudBackupRestore, TestNumericRangeQuery64, SOLR749Test, ChaosMonkeySafeLeaderWithPullReplicasTest, SearchHandlerTest, TestOverriddenPrefixQueryForCustomFieldType, TestCorePropertiesReload, TestTestInjection, HdfsRestartWhileUpdatingTest, TestDistribStateManager]
[junit4] Completed [148/768 (1!)] on J1 in 32.53s, 5 tests, 1 failure <<< FAILURES!
[...truncated 1515 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/init-core-data-001
[junit4] 2> 2938983 WARN (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=21 numCloses=21
[junit4] 2> 2938983 INFO (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2938984 INFO (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2938985 INFO (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2938985 INFO (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.c.a.s.TestClusterStateProvider ####### Using simulated components? false
[junit4] 2> 2938985 INFO (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/tempDir-001
[junit4] 2> 2938986 INFO (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2938986 INFO (Thread-9328) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2938986 INFO (Thread-9328) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2938990 ERROR (Thread-9328) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2939086 INFO (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:33135
[junit4] 2> 2939090 INFO (zkConnectionManagerCallback-7242-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939097 INFO (jetty-launcher-7239-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2939097 INFO (jetty-launcher-7239-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2939097 INFO (jetty-launcher-7239-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2939098 INFO (jetty-launcher-7239-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@47a901{/solr,null,AVAILABLE}
[junit4] 2> 2939098 INFO (jetty-launcher-7239-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1c9c192{/solr,null,AVAILABLE}
[junit4] 2> 2939098 INFO (jetty-launcher-7239-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@123127c{/solr,null,AVAILABLE}
[junit4] 2> 2939101 INFO (jetty-launcher-7239-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@714b0d{SSL,[ssl, http/1.1]}{127.0.0.1:34557}
[junit4] 2> 2939101 INFO (jetty-launcher-7239-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1ca55cf{SSL,[ssl, http/1.1]}{127.0.0.1:41357}
[junit4] 2> 2939101 INFO (jetty-launcher-7239-thread-3) [ ] o.e.j.s.Server Started @2941121ms
[junit4] 2> 2939101 INFO (jetty-launcher-7239-thread-1) [ ] o.e.j.s.Server Started @2941121ms
[junit4] 2> 2939101 INFO (jetty-launcher-7239-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=34557}
[junit4] 2> 2939101 INFO (jetty-launcher-7239-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41357}
[junit4] 2> 2939101 INFO (jetty-launcher-7239-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@16ea5c3{SSL,[ssl, http/1.1]}{127.0.0.1:38081}
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-2) [ ] o.e.j.s.Server Started @2941121ms
[junit4] 2> 2939102 ERROR (jetty-launcher-7239-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=38081}
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2939102 ERROR (jetty-launcher-7239-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T11:43:25.716Z
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2939102 ERROR (jetty-launcher-7239-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T11:43:25.716Z
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2939102 INFO (jetty-launcher-7239-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2939103 INFO (jetty-launcher-7239-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2939103 INFO (jetty-launcher-7239-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T11:43:25.717Z
[junit4] 2> 2939103 INFO (zkConnectionManagerCallback-7244-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939104 INFO (zkConnectionManagerCallback-7248-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939104 INFO (zkConnectionManagerCallback-7246-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939104 INFO (jetty-launcher-7239-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2939104 INFO (jetty-launcher-7239-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2939104 INFO (jetty-launcher-7239-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2939128 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f9a199a10002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2939131 INFO (jetty-launcher-7239-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33135/solr
[junit4] 2> 2939131 INFO (jetty-launcher-7239-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33135/solr
[junit4] 2> 2939131 INFO (jetty-launcher-7239-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33135/solr
[junit4] 2> 2939131 INFO (zkConnectionManagerCallback-7256-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939131 INFO (zkConnectionManagerCallback-7260-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939132 INFO (zkConnectionManagerCallback-7258-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939132 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f9a199a10005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2939138 INFO (zkConnectionManagerCallback-7264-thread-1-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939138 INFO (zkConnectionManagerCallback-7262-thread-1-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939139 INFO (zkConnectionManagerCallback-7266-thread-1-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939181 INFO (jetty-launcher-7239-thread-1) [n:127.0.0.1:41357_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2939181 INFO (jetty-launcher-7239-thread-3) [n:127.0.0.1:34557_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2939181 INFO (jetty-launcher-7239-thread-2) [n:127.0.0.1:38081_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2939181 INFO (jetty-launcher-7239-thread-1) [n:127.0.0.1:41357_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41357_solr
[junit4] 2> 2939182 INFO (jetty-launcher-7239-thread-3) [n:127.0.0.1:34557_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34557_solr
[junit4] 2> 2939182 INFO (jetty-launcher-7239-thread-2) [n:127.0.0.1:38081_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:38081_solr
[junit4] 2> 2939182 INFO (jetty-launcher-7239-thread-1) [n:127.0.0.1:41357_solr ] o.a.s.c.Overseer Overseer (id=99353664264667143-127.0.0.1:41357_solr-n_0000000000) starting
[junit4] 2> 2939183 INFO (zkCallback-7265-thread-1-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2939183 INFO (zkCallback-7263-thread-1-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2939183 INFO (zkCallback-7261-thread-1-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2939184 INFO (zkCallback-7265-thread-1-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2939185 INFO (zkCallback-7263-thread-2-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2939185 INFO (zkCallback-7261-thread-1-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2939195 INFO (jetty-launcher-7239-thread-1) [n:127.0.0.1:41357_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41357_solr
[junit4] 2> 2939195 INFO (zkCallback-7265-thread-1-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2939195 INFO (zkCallback-7263-thread-2-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2939201 INFO (zkCallback-7261-thread-1-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2939278 INFO (jetty-launcher-7239-thread-1) [n:127.0.0.1:41357_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41357.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2939279 INFO (jetty-launcher-7239-thread-3) [n:127.0.0.1:34557_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34557.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2939287 INFO (jetty-launcher-7239-thread-1) [n:127.0.0.1:41357_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41357.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2939287 INFO (jetty-launcher-7239-thread-1) [n:127.0.0.1:41357_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41357.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2939288 INFO (jetty-launcher-7239-thread-3) [n:127.0.0.1:34557_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34557.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2939289 INFO (jetty-launcher-7239-thread-1) [n:127.0.0.1:41357_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/tempDir-001/node1/.
[junit4] 2> 2939289 INFO (jetty-launcher-7239-thread-3) [n:127.0.0.1:34557_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34557.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2939290 INFO (jetty-launcher-7239-thread-3) [n:127.0.0.1:34557_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/tempDir-001/node3/.
[junit4] 2> 2939361 INFO (jetty-launcher-7239-thread-2) [n:127.0.0.1:38081_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38081.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2939366 INFO (jetty-launcher-7239-thread-2) [n:127.0.0.1:38081_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38081.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2939366 INFO (jetty-launcher-7239-thread-2) [n:127.0.0.1:38081_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38081.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2939367 INFO (jetty-launcher-7239-thread-2) [n:127.0.0.1:38081_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/tempDir-001/node2/.
[junit4] 2> 2939388 INFO (zkConnectionManagerCallback-7274-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939390 INFO (zkConnectionManagerCallback-7278-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2939390 INFO (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2939391 INFO (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:33135/solr ready
[junit4] 2> 2939434 INFO (qtp7337814-31285) [n:127.0.0.1:38081_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=1&replicationFactor=2&name=.system&nrtReplicas=2&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2939436 INFO (OverseerThreadFactory-10776-thread-1-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.CreateCollectionCmd Create collection .system
[junit4] 2> 2939541 INFO (OverseerStateUpdate-99353664264667143-127.0.0.1:41357_solr-n_0000000000) [n:127.0.0.1:41357_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:34557/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2939541 INFO (OverseerStateUpdate-99353664264667143-127.0.0.1:41357_solr-n_0000000000) [n:127.0.0.1:41357_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:38081/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2939542 INFO (OverseerStateUpdate-99353664264667143-127.0.0.1:41357_solr-n_0000000000) [n:127.0.0.1:41357_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_p4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:41357/solr",
[junit4] 2> "type":"PULL",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2939747 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n2&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2939747 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2939793 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=.system&newCollection=true&name=.system_shard1_replica_p4&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=PULL
[junit4] 2> 2939793 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2939801 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n1&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2939802 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2939905 INFO (zkCallback-7263-thread-1-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2939905 INFO (zkCallback-7265-thread-1-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2939905 INFO (zkCallback-7265-thread-2-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2939905 INFO (zkCallback-7263-thread-2-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2939905 INFO (zkCallback-7261-thread-1-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2939905 INFO (zkCallback-7261-thread-2-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2940767 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2940770 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 2940773 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.IndexSchema [.system_shard1_replica_n2] Schema name=_system collection or core
[junit4] 2> 2940777 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 2940778 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/.system/managed-schema
[junit4] 2> 2940779 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/.system/schema.xml to /configs/.system/schema.xml.bak
[junit4] 2> 2940779 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_n2' using configuration from collection .system, trusted=true
[junit4] 2> 2940780 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_38081.solr.core..system.shard1.replica_n2' (registry 'solr.core..system.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2940780 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2940781 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore [[.system_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/tempDir-001/node2/.system_shard1_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/tempDir-001/node2/./.system_shard1_replica_n2/data/]
[junit4] 2> 2940801 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2940808 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.IndexSchema [.system_shard1_replica_p4] Schema name=_system collection or core
[junit4] 2> 2940809 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2940812 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 2940812 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_p4' using configuration from collection .system, trusted=true
[junit4] 2> 2940813 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41357.solr.core..system.shard1.replica_p4' (registry 'solr.core..system.shard1.replica_p4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2940813 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2940813 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [[.system_shard1_replica_p4] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/tempDir-001/node1/.system_shard1_replica_p4], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/tempDir-001/node1/./.system_shard1_replica_p4/data/]
[junit4] 2> 2940814 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.IndexSchema [.system_shard1_replica_n1] Schema name=_system collection or core
[junit4] 2> 2940817 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 2940817 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_n1' using configuration from collection .system, trusted=true
[junit4] 2> 2940817 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_34557.solr.core..system.shard1.replica_n1' (registry 'solr.core..system.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2940817 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2940817 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [[.system_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/tempDir-001/node3/.system_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/tempDir-001/node3/./.system_shard1_replica_n1/data/]
[junit4] 2> 2940836 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2940836 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2940838 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: if 1 uncommited docs
[junit4] 2> 2940838 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2940839 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ecdfd[.system_shard1_replica_n2] main]
[junit4] 2> 2940840 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/.system
[junit4] 2> 2940861 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/.system
[junit4] 2> 2940861 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/.system/managed-schema
[junit4] 2> 2940862 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 2940862 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2940863 INFO (searcherExecutor-10783-thread-1-processing-n:127.0.0.1:38081_solr x:.system_shard1_replica_n2 s:shard1 c:.system r:core_node5) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore [.system_shard1_replica_n2] Registered new searcher Searcher@1ecdfd[.system_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2940864 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589658630202851328
[junit4] 2> 2940867 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2940873 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2940873 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2940874 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if 1 uncommited docs
[junit4] 2> 2940875 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2940876 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1a7f8f8[.system_shard1_replica_n1] main]
[junit4] 2> 2940877 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.CommitTracker Hard AutoCommit: if 1 uncommited docs
[junit4] 2> 2940877 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2940877 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/.system
[junit4] 2> 2940877 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/.system
[junit4] 2> 2940878 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/.system/managed-schema
[junit4] 2> 2940878 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 2940878 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.SolrIndexSearcher Opening [Searcher@ecc0c[.system_shard1_replica_p4] main]
[junit4] 2> 2940879 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2940879 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/.system
[junit4] 2> 2940879 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/.system
[junit4] 2> 2940879 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/.system/managed-schema
[junit4] 2> 2940879 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 2940879 INFO (searcherExecutor-10785-thread-1-processing-n:127.0.0.1:34557_solr x:.system_shard1_replica_n1 s:shard1 c:.system r:core_node3) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [.system_shard1_replica_n1] Registered new searcher Searcher@1a7f8f8[.system_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2940879 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589658630218579968
[junit4] 2> 2940880 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2940880 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.ZkController .system_shard1_replica_p4 starting background replication from leader
[junit4] 2> 2940880 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 2940880 INFO (searcherExecutor-10784-thread-1-processing-n:127.0.0.1:41357_solr x:.system_shard1_replica_p4 s:shard1 c:.system r:core_node6) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [.system_shard1_replica_p4] Registered new searcher Searcher@ecc0c[.system_shard1_replica_p4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2940882 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 2940882 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2940968 INFO (zkCallback-7263-thread-2-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2940968 INFO (zkCallback-7265-thread-2-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2940968 INFO (zkCallback-7261-thread-2-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2940968 INFO (zkCallback-7261-thread-1-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2940968 INFO (zkCallback-7265-thread-1-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2940968 INFO (zkCallback-7263-thread-1-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941368 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2941368 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2941368 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:38081/solr/.system_shard1_replica_n2/
[junit4] 2> 2941368 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=.system_shard1_replica_n2 url=https://127.0.0.1:38081/solr START replicas=[https://127.0.0.1:34557/solr/.system_shard1_replica_n1/] nUpdates=100
[junit4] 2> 2941372 INFO (qtp30408045-31289) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.S.Request [.system_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2941669 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=.system_shard1_replica_n2 url=https://127.0.0.1:38081/solr DONE. We have no versions. sync failed.
[junit4] 2> 2941669 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2941669 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2941669 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2941671 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:38081/solr/.system_shard1_replica_n2/ shard1
[junit4] 2> 2941771 INFO (zkCallback-7263-thread-1-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941772 INFO (zkCallback-7265-thread-1-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941772 INFO (zkCallback-7261-thread-2-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941772 INFO (zkCallback-7261-thread-1-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941771 INFO (zkCallback-7263-thread-2-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941771 INFO (zkCallback-7265-thread-2-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941783 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Replica core_node5 is leader but it's state is down, skipping replication
[junit4] 2> 2941821 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2941875 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n2&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2128
[junit4] 2> 2941883 INFO (qtp2124465-31283) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=.system&newCollection=true&name=.system_shard1_replica_p4&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=PULL} status=0 QTime=2090
[junit4] 2> 2941887 INFO (qtp30408045-31286) [n:127.0.0.1:34557_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n1&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2085
[junit4] 2> 2941890 INFO (qtp7337814-31285) [n:127.0.0.1:38081_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2941984 INFO (zkCallback-7265-thread-2-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941984 INFO (zkCallback-7261-thread-2-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941984 INFO (zkCallback-7261-thread-1-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941984 INFO (zkCallback-7263-thread-1-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941984 INFO (zkCallback-7263-thread-2-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2941984 INFO (zkCallback-7265-thread-1-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2942890 INFO (qtp7337814-31285) [n:127.0.0.1:38081_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=1&replicationFactor=2&name=.system&nrtReplicas=2&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=3456
[junit4] 2> 2942894 INFO (qtp7337814-31288) [n:127.0.0.1:38081_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2942896 INFO (qtp30408045-31289) [n:127.0.0.1:34557_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2942899 INFO (qtp2124465-31304) [n:127.0.0.1:41357_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 2942990 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAddRemoveNode
[junit4] 2> 2942991 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2942992 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1770ce2{/solr,null,AVAILABLE}
[junit4] 2> 2942992 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1b49fb3{SSL,[ssl, http/1.1]}{127.0.0.1:46059}
[junit4] 2> 2942992 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.e.j.s.Server Started @2945011ms
[junit4] 2> 2942992 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46059}
[junit4] 2> 2942992 ERROR (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2942992 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 2942992 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2942993 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2942993 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T11:43:29.607Z
[junit4] 2> 2942994 INFO (zkConnectionManagerCallback-7280-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2942994 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2943003 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:33135/solr
[junit4] 2> 2943004 INFO (zkConnectionManagerCallback-7284-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2943006 INFO (zkConnectionManagerCallback-7286-thread-1-processing-n:127.0.0.1:46059_solr) [n:127.0.0.1:46059_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 2943008 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [n:127.0.0.1:46059_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2943009 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [n:127.0.0.1:46059_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2943010 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [n:127.0.0.1:46059_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46059_solr
[junit4] 2> 2943011 INFO (zkCallback-7265-thread-1-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2943011 INFO (zkCallback-7277-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2943011 INFO (zkCallback-7263-thread-2-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2943011 INFO (zkCallback-7261-thread-1-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2943014 INFO (zkCallback-7285-thread-1-processing-n:127.0.0.1:46059_solr) [n:127.0.0.1:46059_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2943098 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [n:127.0.0.1:46059_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46059.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2943108 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [n:127.0.0.1:46059_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46059.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2943108 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [n:127.0.0.1:46059_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46059.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@15adfc9
[junit4] 2> 2943109 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [n:127.0.0.1:46059_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001/tempDir-001/node4/.
[junit4] 2> 2943439 INFO (OverseerCollectionConfigSetProcessor-99353664264667143-127.0.0.1:41357_solr-n_0000000000) [n:127.0.0.1:41357_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2943882 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication
[junit4] 2> 2943882 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Updated masterUrl to https://127.0.0.1:38081/solr/.system_shard1_replica_n2/
[junit4] 2> 2943883 INFO (qtp7337814-31306) [n:127.0.0.1:38081_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.S.Request [.system_shard1_replica_n2] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2943883 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2943883 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2943883 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2943883 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 2943883 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 2943883 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2943883 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@cf5778 commitCommandVersion:0
[junit4] 2> 2943884 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.SolrIndexSearcher Opening [Searcher@ac5dd0[.system_shard1_replica_p4] main]
[junit4] 2> 2943884 INFO (indexFetcher-10798-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2943885 INFO (searcherExecutor-10784-thread-1-processing-n:127.0.0.1:41357_solr x:.system_shard1_replica_p4 s:shard1 c:.system r:core_node6) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [.system_shard1_replica_p4] Registered new searcher Searcher@ac5dd0[.system_shard1_replica_p4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2945148 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1ca55cf{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 2945149 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=31351308
[junit4] 2> 2945149 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2945149 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6f8dd6: rootName = solr_41357, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1fb80da
[junit4] 2> 2945158 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2945158 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@92a5cf: rootName = solr_41357, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@2d5ed4
[junit4] 2> 2945163 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2945163 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@d0e96a: rootName = solr_41357, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@1b56b46
[junit4] 2> 2945164 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:41357_solr
[junit4] 2> 2945164 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[BBF0E0A8DF27CA36]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 2945164 INFO (zkCallback-7285-thread-1-processing-n:127.0.0.1:46059_solr) [n:127.0.0.1:46059_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 2945164 INFO (zkCallback-7277-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 2945164 INFO (zkCallback-7263-thread-2-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 2945164 INFO (zkCallback-7265-thread-1-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 2945165 INFO (zkCallback-7261-thread-2-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2945165 INFO (zkCallback-7263-thread-3-processing-n:127.0.0.1:41357_solr) [n:127.0.0.1:41357_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2945165 INFO (zkCallback-7265-thread-3-processing-n:127.0.0.1:34557_solr) [n:127.0.0.1:34557_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2945165 INFO (coreCloseExecutor-10805-thread-1) [n:127.0.0.1:41357_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [.system_shard1_replica_p4] CLOSING SolrCore org.apache.solr.core.SolrCore@1bf824
[junit4] 2> 2945165 INFO (zkCallback-7261-thread-3-processing-n:127.0.0.1:38081_solr) [n:127.0.0.1:38081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedE
[...truncated too long message...]
4)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:364)
[junit4] 2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:879)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:704)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:703)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$900(ZkStateReader.java:72)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1090)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1085)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:268)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2948783 INFO (jetty-closer-7240-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1770ce2{/solr,null,UNAVAILABLE}
[junit4] 2> 2948784 ERROR (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2948784 INFO (SUITE-TestClusterStateProvider-seed#[BBF0E0A8DF27CA36]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33135 33135
[junit4] 2> 2954208 INFO (Thread-9328) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:33135 33135
[junit4] 2> 2954209 WARN (Thread-9328) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 3 /solr/configs/.system
[junit4] 2> 3 /solr/configs/.system/managed-schema
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 23 /solr/collections/.system/state.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 3 /solr/autoscaling.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_BBF0E0A8DF27CA36-001
[junit4] 2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=3907, maxDocsPerChunk=2, blockSize=348), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=3907, blockSize=348)), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@1148d55), locale=es-DO, timezone=America/North_Dakota/New_Salem
[junit4] 2> NOTE: Linux 4.13.0-26-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=169977912,total=521928704
[junit4] 2> NOTE: All tests run in this JVM: [TestCollectionAPIs, TestConfigOverlay, TestNamedUpdateProcessors, ActionThrottleTest, TestMaxTokenLenTokenizer, TestFastOutputStream, LeaderElectionContextKeyTest, TestFieldCache, TestMaxScoreQueryParser, HdfsDirectoryFactoryTest, TestPullReplica, SpatialFilterTest, TestDistribIDF, TestSchemaSimilarityResource, SolrJmxReporterCloudTest, TestComputePlanAction, TestRecovery, TestConfig, TestUniqueKeyFieldResource, TestComponentsName, ZkControllerTest, TestTriggerIntegration, AutoAddReplicasIntegrationTest, TestSQLHandler, TestRandomDVFaceting, TestPolicyCloud, AssignBackwardCompatibilityTest, URLClassifyProcessorTest, BasicFunctionalityTest, TestReplicationHandler, TestTrieFacet, TestIndexingPerformance, TestCloudDeleteByQuery, TestCloudPivotFacet, TestCSVLoader, SoftAutoCommitTest, SolrIndexMetricsTest, SolrCoreMetricManagerTest, SpatialRPTFieldTypeTest, AnalyticsQueryTest, TestHdfsCloudBackupRestore, DirectUpdateHandlerOptimizeTest, BlockCacheTest, TestDistributedSearch, TestNodeLostTrigger, TestSmileRequest, TestCloudSearcherWarming, UpdateRequestProcessorFactoryTest, TestPhraseSuggestions, RecoveryAfterSoftCommitTest, SolrInfoBeanTest, TestSort, TestMergePolicyConfig, V2StandaloneTest, ZkStateReaderTest, JavabinLoaderTest, MetricUtilsTest, TestSegmentSorting, SolrMetricsIntegrationTest, TestCrossCoreJoin, DistributedMLTComponentTest, TestCodecSupport, TestDistribDocBasedVersion, PeerSyncTest, TestRequestForwarding, TestImpersonationWithHadoopAuth, TestSolrIndexConfig, PrimitiveFieldTypeTest, MetricsHandlerTest, TestFieldCollectionResource, TestUtils, TestComplexPhraseLeadingWildcard, TestEmbeddedSolrServerSchemaAPI, CheckHdfsIndexTest, TestJsonRequest, DateMathParserTest, SolrCloudReportersTest, SliceStateTest, TestMiniSolrCloudClusterSSL, BJQParserTest, TestReplicationHandlerBackup, NodeMutatorTest, TestUseDocValuesAsStored, CleanupOldIndexTest, TestNumericRangeQuery32, TestDFRSimilarityFactory, DistributedQueryComponentCustomSortTest, HttpSolrCallGetCoreTest, SearchHandlerTest, LargeFieldTest, HLLSerializationTest, ForceLeaderTest, UUIDUpdateProcessorFallbackTest, OverseerCollectionConfigSetProcessorTest, TestGenericDistributedQueue, MigrateRouteKeyTest, TestSolrConfigHandlerConcurrent, TestSolrCloudWithSecureImpersonation, TestLeaderInitiatedRecoveryThread, TestFieldTypeResource, ConnectionManagerTest, SharedFSAutoReplicaFailoverTest, TestFieldSortValues, DocValuesMissingTest, NodeLostTriggerTest, TestPointFields, TestSolrXml, QueryParsingTest, SyncSliceTest, UpdateLogTest, TestTolerantUpdateProcessorRandomCloud, ExternalFileFieldSortTest, UpdateParamsTest, SuggestComponentTest, TestReloadDeadlock, StatelessScriptUpdateProcessorFactoryTest, TestConfigSets, TestReversedWildcardFilterFactory, HdfsCollectionsAPIDistributedZkTest, HdfsThreadLeakTest, TestShardHandlerFactory, DeleteInactiveReplicaTest, DocExpirationUpdateProcessorFactoryTest, SimpleFacetsTest, TestMinMaxOnMultiValuedField, ShardRoutingCustomTest, TestDistributedGrouping, TestHashPartitioner, DistributedSpellCheckComponentTest, TermVectorComponentDistributedTest, TestStressReorder, TestJoin, TestCoreContainer, SolrCoreTest, StatsComponentTest, SpellCheckComponentTest, QueryElevationComponentTest, ConvertedLegacyTest, TestFunctionQuery, DirectUpdateHandlerTest, TestBadConfig, HighlighterTest, ShowFileRequestHandlerTest, DistributedQueryElevationComponentTest, SolrIndexSplitterTest, TestCoreDiscovery, AnalysisAfterCoreReloadTest, SpellCheckCollatorTest, SolrRequestParserTest, TestFoldingMultitermQuery, DocValuesTest, TestStressLucene, SuggesterWFSTTest, NoCacheHeaderTest, SchemaVersionSpecificBehaviorTest, TestUpdate, FieldMutatingUpdateProcessorTest, TestWordDelimiterFilterFactory, SortByFunctionTest, TestRemoteStreaming, TestSolrDeletionPolicy1, CacheHeaderTest, LukeRequestHandlerTest, TestQueryUtils, DirectSolrSpellCheckerTest, TestQueryTypes, TermsComponentTest, DocumentBuilderTest, FieldAnalysisRequestHandlerTest, FastVectorHighlighterTest, LoggingHandlerTest, IndexSchemaTest, ReturnFieldsTest, MBeansHandlerTest, HighlighterConfigTest, CopyFieldTest, MultiTermTest, TestDocSet, TestBinaryField, NumericFieldsTest, MinimalSchemaTest, TestFuzzyAnalyzedSuggestions, SpellPossibilityIteratorTest, EchoParamsTest, TestPerFieldSimilarity, TestLMDirichletSimilarityFactory, TestIBSimilarityFactory, TimeZoneUtilsTest, OpenExchangeRatesOrgProviderTest, PreAnalyzedFieldTest, TestSuggestSpellingConverter, SpellingQueryConverterTest, TestSolrJ, UUIDFieldTest, CursorPagingTest, TestCursorMarkWithoutUniqueKey, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, TestEmbeddedSolrServerAdminHandler, ConnectionReuseTest, AssignTest, CollectionStateFormat2Test, CollectionsAPISolrJTest, DeleteLastCustomShardedReplicaTest, DistribCursorPagingTest, DistribJoinFromCollectionTest, DistributedQueueTest, DocValuesNotIndexedTest, LeaderFailureAfterFreshStartTest, LeaderInitiatedRecoveryOnCommitTest, MissingSegmentRecoveryTest, MoveReplicaHDFSTest, OverseerModifyCollectionTest, OverseerRolesTest, OverseerStatusTest, ReplaceNodeTest, SolrCLIZkUtilsTest, SolrCloudExampleTest, SolrXmlInZkTest, TestAuthenticationFramework, TestCloudInspectUtil, TestCloudPseudoReturnFields, TestCloudRecovery, TestCollectionAPI, TestCollectionsAPIViaSolrCloudCluster, TestConfigSetsAPI, TestConfigSetsAPIExclusivity, TestConfigSetsAPIZkFailure, TestDeleteCollectionOnDownNodes, TestDownShardTolerantSearch, TestLeaderElectionZkExpiry, TestPullReplicaErrorHandling, TestRequestStatusCollectionAPI, TestSolrCloudWithKerberosAlt, TestStressInPlaceUpdates, HdfsAutoAddReplicasIntegrationTest, TestClusterStateProvider]
[junit4] Completed [585/768 (2!)] on J0 in 15.33s, 2 tests, 1 failure <<< FAILURES!
[...truncated 44447 lines...]
[JENKINS] Lucene-Solr-master-Linux (64bit/jdk-9.0.1) - Build #
21276 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21276/
Java: 64bit/jdk-9.0.1 -XX:-UseCompressedOops -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig
Error Message:
expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{ "cluster-preferences":[{"maximize":"freedisk"}], "triggers":{".auto_add_replicas":{ "name":".auto_add_replicas", "event":"nodeLost", "waitFor":30, "actions":[ { "name":"auto_add_replicas_plan", "class":"solr.AutoAddReplicasPlanAction"}, { "name":"execute_plan", "class":"solr.ExecutePlanAction"}], "enabled":true}}, "listeners":{".auto_add_replicas.system":{ "trigger":".auto_add_replicas", "afterAction":[], "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED", "BEFORE_ACTION", "AFTER_ACTION", "IGNORED"], "class":"org.apache.solr.cloud.autoscaling.SystemLogListener", "beforeAction":[]}}, "properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{ "cluster-preferences":[{"maximize":"freedisk"}], "triggers":{".auto_add_replicas":{ "name":".auto_add_replicas", "event":"nodeLost", "waitFor":30, "actions":[ { "name":"auto_add_replicas_plan", "class":"solr.AutoAddReplicasPlanAction"}, { "name":"execute_plan", "class":"solr.ExecutePlanAction"}], "enabled":true}}, "listeners":{".auto_add_replicas.system":{ "trigger":".auto_add_replicas", "afterAction":[], "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED", "BEFORE_ACTION", "AFTER_ACTION", "IGNORED"], "class":"org.apache.solr.cloud.autoscaling.SystemLogListener", "beforeAction":[]}}, "properties":{}}>
Stack Trace:
java.lang.AssertionError: expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
"cluster-preferences":[{"maximize":"freedisk"}],
"triggers":{".auto_add_replicas":{
"name":".auto_add_replicas",
"event":"nodeLost",
"waitFor":30,
"actions":[
{
"name":"auto_add_replicas_plan",
"class":"solr.AutoAddReplicasPlanAction"},
{
"name":"execute_plan",
"class":"solr.ExecutePlanAction"}],
"enabled":true}},
"listeners":{".auto_add_replicas.system":{
"trigger":".auto_add_replicas",
"afterAction":[],
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED",
"BEFORE_ACTION",
"AFTER_ACTION",
"IGNORED"],
"class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
"beforeAction":[]}},
"properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
"cluster-preferences":[{"maximize":"freedisk"}],
"triggers":{".auto_add_replicas":{
"name":".auto_add_replicas",
"event":"nodeLost",
"waitFor":30,
"actions":[
{
"name":"auto_add_replicas_plan",
"class":"solr.AutoAddReplicasPlanAction"},
{
"name":"execute_plan",
"class":"solr.ExecutePlanAction"}],
"enabled":true}},
"listeners":{".auto_add_replicas.system":{
"trigger":".auto_add_replicas",
"afterAction":[],
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED",
"BEFORE_ACTION",
"AFTER_ACTION",
"IGNORED"],
"class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
"beforeAction":[]}},
"properties":{}}>
at __randomizedtesting.SeedInfo.seed([956EAD68A618714B:AAE6AEC0B14E81AC]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig(TestClusterStateProvider.java:214)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 12117 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/init-core-data-001
[junit4] 2> 481100 INFO (SUITE-TestClusterStateProvider-seed#[956EAD68A618714B]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 481100 INFO (SUITE-TestClusterStateProvider-seed#[956EAD68A618714B]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", ssl=0.0/0.0, value=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 481101 INFO (SUITE-TestClusterStateProvider-seed#[956EAD68A618714B]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 481101 INFO (SUITE-TestClusterStateProvider-seed#[956EAD68A618714B]-worker) [ ] o.a.s.c.a.s.TestClusterStateProvider ####### Using simulated components? true
[junit4] 2> 481101 INFO (SUITE-TestClusterStateProvider-seed#[956EAD68A618714B]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/tempDir-001
[junit4] 2> 481101 INFO (SUITE-TestClusterStateProvider-seed#[956EAD68A618714B]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 481101 INFO (Thread-515) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 481101 INFO (Thread-515) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 481102 ERROR (Thread-515) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 481201 INFO (SUITE-TestClusterStateProvider-seed#[956EAD68A618714B]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:35371
[junit4] 2> 481203 INFO (zkConnectionManagerCallback-872-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481205 INFO (jetty-launcher-869-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 481205 INFO (jetty-launcher-869-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 481205 INFO (jetty-launcher-869-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 481205 INFO (jetty-launcher-869-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2a2ce3be{/solr,null,AVAILABLE}
[junit4] 2> 481205 INFO (jetty-launcher-869-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@26b21342{/solr,null,AVAILABLE}
[junit4] 2> 481206 INFO (jetty-launcher-869-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5b954d4f{/solr,null,AVAILABLE}
[junit4] 2> 481206 INFO (jetty-launcher-869-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@127fb04a{SSL,[ssl, http/1.1]}{127.0.0.1:45667}
[junit4] 2> 481206 INFO (jetty-launcher-869-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@779f1db0{SSL,[ssl, http/1.1]}{127.0.0.1:41443}
[junit4] 2> 481206 INFO (jetty-launcher-869-thread-1) [ ] o.e.j.s.Server Started @483405ms
[junit4] 2> 481206 INFO (jetty-launcher-869-thread-2) [ ] o.e.j.s.Server Started @483406ms
[junit4] 2> 481206 INFO (jetty-launcher-869-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45667}
[junit4] 2> 481206 INFO (jetty-launcher-869-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41443}
[junit4] 2> 481206 INFO (jetty-launcher-869-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7f0eb71e{SSL,[ssl, http/1.1]}{127.0.0.1:46393}
[junit4] 2> 481206 INFO (jetty-launcher-869-thread-3) [ ] o.e.j.s.Server Started @483406ms
[junit4] 2> 481206 INFO (jetty-launcher-869-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=46393}
[junit4] 2> 481207 ERROR (jetty-launcher-869-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 481207 INFO (jetty-launcher-869-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 481207 INFO (jetty-launcher-869-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 481207 INFO (jetty-launcher-869-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 481207 INFO (jetty-launcher-869-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T06:00:44.837243Z
[junit4] 2> 481207 ERROR (jetty-launcher-869-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 481207 INFO (jetty-launcher-869-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 481207 INFO (jetty-launcher-869-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 481207 INFO (jetty-launcher-869-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 481207 INFO (jetty-launcher-869-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T06:00:44.837901Z
[junit4] 2> 481208 INFO (zkConnectionManagerCallback-874-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481208 ERROR (jetty-launcher-869-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 481208 INFO (jetty-launcher-869-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 481208 INFO (jetty-launcher-869-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 481208 INFO (jetty-launcher-869-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 481208 INFO (jetty-launcher-869-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 481208 INFO (jetty-launcher-869-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T06:00:44.838585Z
[junit4] 2> 481208 INFO (zkConnectionManagerCallback-876-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481209 INFO (jetty-launcher-869-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 481213 INFO (jetty-launcher-869-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35371/solr
[junit4] 2> 481213 INFO (jetty-launcher-869-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35371/solr
[junit4] 2> 481218 INFO (zkConnectionManagerCallback-878-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481218 INFO (zkConnectionManagerCallback-884-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481218 INFO (zkConnectionManagerCallback-886-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481218 INFO (jetty-launcher-869-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 481219 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f867ddbc0005, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> 481220 INFO (zkConnectionManagerCallback-889-thread-1-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481220 INFO (zkConnectionManagerCallback-890-thread-1-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481235 INFO (jetty-launcher-869-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35371/solr
[junit4] 2> 481236 INFO (zkConnectionManagerCallback-896-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481238 INFO (zkConnectionManagerCallback-898-thread-1-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481243 INFO (jetty-launcher-869-thread-2) [n:127.0.0.1:41443_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 481243 INFO (jetty-launcher-869-thread-2) [n:127.0.0.1:41443_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41443_solr
[junit4] 2> 481244 INFO (jetty-launcher-869-thread-2) [n:127.0.0.1:41443_solr ] o.a.s.c.Overseer Overseer (id=99352316787556361-127.0.0.1:41443_solr-n_0000000000) starting
[junit4] 2> 481261 INFO (jetty-launcher-869-thread-3) [n:127.0.0.1:46393_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 481261 INFO (jetty-launcher-869-thread-1) [n:127.0.0.1:45667_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 481261 INFO (jetty-launcher-869-thread-3) [n:127.0.0.1:46393_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46393_solr
[junit4] 2> 481262 INFO (OverseerStateUpdate-99352316787556361-127.0.0.1:41443_solr-n_0000000000) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 481264 INFO (jetty-launcher-869-thread-1) [n:127.0.0.1:45667_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45667_solr
[junit4] 2> 481264 INFO (jetty-launcher-869-thread-2) [n:127.0.0.1:41443_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41443_solr
[junit4] 2> 481279 INFO (zkCallback-887-thread-1-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 481285 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (3)
[junit4] 2> 481289 INFO (zkCallback-888-thread-1-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 481308 INFO (jetty-launcher-869-thread-3) [n:127.0.0.1:46393_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46393.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 481311 INFO (jetty-launcher-869-thread-1) [n:127.0.0.1:45667_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45667.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 481315 INFO (jetty-launcher-869-thread-3) [n:127.0.0.1:46393_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46393.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 481315 INFO (jetty-launcher-869-thread-3) [n:127.0.0.1:46393_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46393.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 481316 INFO (jetty-launcher-869-thread-3) [n:127.0.0.1:46393_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/tempDir-001/node3/.
[junit4] 2> 481319 INFO (jetty-launcher-869-thread-2) [n:127.0.0.1:41443_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41443.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 481319 INFO (jetty-launcher-869-thread-1) [n:127.0.0.1:45667_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45667.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 481320 INFO (jetty-launcher-869-thread-1) [n:127.0.0.1:45667_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45667.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 481321 INFO (jetty-launcher-869-thread-1) [n:127.0.0.1:45667_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/tempDir-001/node1/.
[junit4] 2> 481326 INFO (jetty-launcher-869-thread-2) [n:127.0.0.1:41443_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41443.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 481326 INFO (jetty-launcher-869-thread-2) [n:127.0.0.1:41443_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41443.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 481327 INFO (jetty-launcher-869-thread-2) [n:127.0.0.1:41443_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/tempDir-001/node2/.
[junit4] 2> 481407 INFO (zkConnectionManagerCallback-904-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481425 INFO (zkConnectionManagerCallback-908-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 481425 INFO (SUITE-TestClusterStateProvider-seed#[956EAD68A618714B]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 481426 INFO (SUITE-TestClusterStateProvider-seed#[956EAD68A618714B]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:35371/solr ready
[junit4] 2> 481441 INFO (qtp1010268621-3071) [n:127.0.0.1:41443_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=1&replicationFactor=2&name=.system&nrtReplicas=2&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 481445 INFO (OverseerThreadFactory-1361-thread-1-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.CreateCollectionCmd Create collection .system
[junit4] 2> 481550 INFO (OverseerStateUpdate-99352316787556361-127.0.0.1:41443_solr-n_0000000000) [n:127.0.0.1:41443_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:41443/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 481551 INFO (OverseerStateUpdate-99352316787556361-127.0.0.1:41443_solr-n_0000000000) [n:127.0.0.1:41443_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45667/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 481552 INFO (OverseerStateUpdate-99352316787556361-127.0.0.1:41443_solr-n_0000000000) [n:127.0.0.1:41443_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_p4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:46393/solr",
[junit4] 2> "type":"PULL",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 481761 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n1&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 481761 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 481769 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=.system&newCollection=true&name=.system_shard1_replica_p4&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=PULL
[junit4] 2> 481769 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 481772 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n2&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 481772 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 481884 INFO (zkCallback-888-thread-1-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 481884 INFO (zkCallback-897-thread-1-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 481884 INFO (zkCallback-897-thread-2-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 481884 INFO (zkCallback-887-thread-1-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 481884 INFO (zkCallback-888-thread-2-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 481884 INFO (zkCallback-887-thread-2-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 482768 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 482770 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 482772 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.IndexSchema [.system_shard1_replica_n1] Schema name=_system collection or core
[junit4] 2> 482774 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 482774 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/.system/managed-schema
[junit4] 2> 482775 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/.system/schema.xml to /configs/.system/schema.xml.bak
[junit4] 2> 482775 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_n1' using configuration from collection .system, trusted=true
[junit4] 2> 482776 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41443.solr.core..system.shard1.replica_n1' (registry 'solr.core..system.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 482776 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 482776 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [[.system_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/tempDir-001/node2/.system_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/tempDir-001/node2/./.system_shard1_replica_n1/data/]
[junit4] 2> 482787 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 482787 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 482806 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.IndexSchema [.system_shard1_replica_p4] Schema name=_system collection or core
[junit4] 2> 482806 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.IndexSchema [.system_shard1_replica_n2] Schema name=_system collection or core
[junit4] 2> 482809 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 482809 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 482809 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_p4' using configuration from collection .system, trusted=true
[junit4] 2> 482809 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_n2' using configuration from collection .system, trusted=true
[junit4] 2> 482810 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_46393.solr.core..system.shard1.replica_p4' (registry 'solr.core..system.shard1.replica_p4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 482810 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45667.solr.core..system.shard1.replica_n2' (registry 'solr.core..system.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 482810 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 482810 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 482810 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [[.system_shard1_replica_p4] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/tempDir-001/node3/.system_shard1_replica_p4], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/tempDir-001/node3/./.system_shard1_replica_p4/data/]
[junit4] 2> 482810 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore [[.system_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/tempDir-001/node1/.system_shard1_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/tempDir-001/node1/./.system_shard1_replica_n2/data/]
[junit4] 2> 482825 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 482825 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 482826 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if 1 uncommited docs
[junit4] 2> 482826 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 482827 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3cba673c[.system_shard1_replica_n1] main]
[junit4] 2> 482828 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/.system
[junit4] 2> 482828 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/.system
[junit4] 2> 482828 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/.system/managed-schema
[junit4] 2> 482828 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 482829 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 482830 INFO (searcherExecutor-1368-thread-1-processing-n:127.0.0.1:41443_solr x:.system_shard1_replica_n1 s:shard1 c:.system r:core_node3) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [.system_shard1_replica_n1] Registered new searcher Searcher@3cba673c[.system_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 482830 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589637070412840960
[junit4] 2> 482833 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 482873 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.CommitTracker Hard AutoCommit: if 1 uncommited docs
[junit4] 2> 482873 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 482874 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 482874 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 482875 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: if 1 uncommited docs
[junit4] 2> 482875 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 482875 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.SolrIndexSearcher Opening [Searcher@4377ebbc[.system_shard1_replica_p4] main]
[junit4] 2> 482876 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7b30ab66[.system_shard1_replica_n2] main]
[junit4] 2> 482876 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/.system
[junit4] 2> 482876 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/.system
[junit4] 2> 482876 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/.system/managed-schema
[junit4] 2> 482876 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 482876 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/.system
[junit4] 2> 482876 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/.system
[junit4] 2> 482876 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/.system/managed-schema
[junit4] 2> 482876 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 482877 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 482877 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 482877 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589637070462124032
[junit4] 2> 482877 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.ZkController .system_shard1_replica_p4 starting background replication from leader
[junit4] 2> 482877 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 482877 INFO (searcherExecutor-1370-thread-1-processing-n:127.0.0.1:46393_solr x:.system_shard1_replica_p4 s:shard1 c:.system r:core_node6) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [.system_shard1_replica_p4] Registered new searcher Searcher@4377ebbc[.system_shard1_replica_p4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 482877 INFO (searcherExecutor-1369-thread-1-processing-n:127.0.0.1:45667_solr x:.system_shard1_replica_n2 s:shard1 c:.system r:core_node5) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore [.system_shard1_replica_n2] Registered new searcher Searcher@7b30ab66[.system_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 482878 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 482878 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 482934 INFO (zkCallback-887-thread-2-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 482934 INFO (zkCallback-897-thread-2-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 482934 INFO (zkCallback-888-thread-2-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 482934 INFO (zkCallback-888-thread-1-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 482934 INFO (zkCallback-897-thread-3-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 482934 INFO (zkCallback-887-thread-1-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483334 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 483334 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 483334 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:41443/solr/.system_shard1_replica_n1/
[junit4] 2> 483334 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=.system_shard1_replica_n1 url=https://127.0.0.1:41443/solr START replicas=[https://127.0.0.1:45667/solr/.system_shard1_replica_n2/] nUpdates=100
[junit4] 2> 483339 INFO (qtp175703866-3064) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.S.Request [.system_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 483634 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=.system_shard1_replica_n1 url=https://127.0.0.1:41443/solr DONE. We have no versions. sync failed.
[junit4] 2> 483634 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 483635 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 483635 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 483637 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:41443/solr/.system_shard1_replica_n1/ shard1
[junit4] 2> 483738 INFO (zkCallback-888-thread-2-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483738 INFO (zkCallback-887-thread-1-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483738 INFO (zkCallback-888-thread-1-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483738 INFO (zkCallback-897-thread-3-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483738 INFO (zkCallback-887-thread-2-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483738 INFO (zkCallback-897-thread-2-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483788 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 483789 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n1&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2028
[junit4] 2> 483880 INFO (qtp196294282-3086) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=.system&newCollection=true&name=.system_shard1_replica_p4&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=PULL} status=0 QTime=2110
[junit4] 2> 483880 INFO (qtp175703866-3062) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n2&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2108
[junit4] 2> 483882 INFO (qtp1010268621-3071) [n:127.0.0.1:41443_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 483981 INFO (zkCallback-888-thread-2-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483981 INFO (zkCallback-887-thread-1-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483981 INFO (zkCallback-888-thread-1-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483981 INFO (zkCallback-897-thread-3-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483981 INFO (zkCallback-887-thread-2-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 483981 INFO (zkCallback-897-thread-2-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 484882 INFO (qtp1010268621-3071) [n:127.0.0.1:41443_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=1&replicationFactor=2&name=.system&nrtReplicas=2&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=3440
[junit4] 2> 484888 INFO (qtp1010268621-3063) [n:127.0.0.1:41443_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 484892 INFO (qtp175703866-3064) [n:127.0.0.1:45667_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 484896 INFO (qtp196294282-3085) [n:127.0.0.1:46393_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 484917 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAddRemoveNode
[junit4] 2> 484919 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 484920 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@757e1e98{/solr,null,AVAILABLE}
[junit4] 2> 484920 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@68ec6a00{SSL,[ssl, http/1.1]}{127.0.0.1:37513}
[junit4] 2> 484921 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.e.j.s.Server Started @487120ms
[junit4] 2> 484921 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=37513}
[junit4] 2> 484921 ERROR (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 484921 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 484921 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 484921 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 484921 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T06:00:48.551601Z
[junit4] 2> 484922 INFO (zkConnectionManagerCallback-910-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 484923 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 484931 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:35371/solr
[junit4] 2> 484931 INFO (zkConnectionManagerCallback-914-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 484933 INFO (zkConnectionManagerCallback-916-thread-1-processing-n:127.0.0.1:37513_solr) [n:127.0.0.1:37513_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 484936 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [n:127.0.0.1:37513_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 484936 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [n:127.0.0.1:37513_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 484938 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [n:127.0.0.1:37513_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37513_solr
[junit4] 2> 484938 INFO (zkCallback-907-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 484939 INFO (zkCallback-887-thread-2-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 484939 INFO (zkCallback-888-thread-2-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 484943 INFO (zkCallback-897-thread-2-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 484943 INFO (zkCallback-915-thread-1-processing-n:127.0.0.1:37513_solr) [n:127.0.0.1:37513_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 484972 INFO (indexFetcher-1383-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Updated masterUrl to https://127.0.0.1:41443/solr/.system_shard1_replica_n1/
[junit4] 2> 484976 INFO (qtp1010268621-3061) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.S.Request [.system_shard1_replica_n1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 484976 INFO (indexFetcher-1383-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 484976 INFO (indexFetcher-1383-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 484976 INFO (indexFetcher-1383-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 484976 INFO (indexFetcher-1383-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 484976 INFO (indexFetcher-1383-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 484977 INFO (indexFetcher-1383-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 484977 INFO (indexFetcher-1383-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@59ff867f commitCommandVersion:0
[junit4] 2> 484978 INFO (indexFetcher-1383-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.SolrIndexSearcher Opening [Searcher@cf98296[.system_shard1_replica_p4] main]
[junit4] 2> 484978 INFO (indexFetcher-1383-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 484978 INFO (searcherExecutor-1370-thread-1-processing-n:127.0.0.1:46393_solr x:.system_shard1_replica_p4 s:shard1 c:.system r:core_node6) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [.system_shard1_replica_p4] Registered new searcher Searcher@cf98296[.system_shard1_replica_p4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 484980 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [n:127.0.0.1:37513_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37513.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 484986 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [n:127.0.0.1:37513_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37513.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 484986 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [n:127.0.0.1:37513_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_37513.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@50858ff0
[junit4] 2> 484987 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [n:127.0.0.1:37513_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001/tempDir-001/node4/.
[junit4] 2> 485218 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@127fb04a{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 485219 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1637485926
[junit4] 2> 485219 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 485219 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@b72d68d: rootName = solr_45667, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@41946c4
[junit4] 2> 485223 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 485223 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6c2871b8: rootName = solr_45667, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7708928b
[junit4] 2> 485226 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 485226 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@bfe4a84: rootName = solr_45667, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@453cff07
[junit4] 2> 485227 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45667_solr
[junit4] 2> 485228 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 485228 INFO (zkCallback-907-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 485228 INFO (zkCallback-887-thread-2-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 485228 INFO (zkCallback-888-thread-2-processing-n:127.0.0.1:45667_solr) [n:127.0.0.1:45667_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 485228 INFO (zkCallback-897-thread-2-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 485228 INFO (zkCallback-915-thread-1-processing-n:127.0.0.1:37513_solr) [n:127.0.0.1:37513_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 485229 INFO (coreCloseExecutor-1394-thread-1) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore [.system_shard1_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@255b23da
[junit4] 2> 485229 INFO (coreCloseExecutor-1394-thread-1) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core..system.shard1.replica_n2, tag=626729946
[junit4] 2> 485229 INFO (coreCloseExecutor-1394-thread-1) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@f4e7bf0: rootName = solr_45667, domain = solr.core..system.shard1.replica_n2, service url = null, agent id = null] for registry solr.core..system.shard1.replica_n2 / com.codahale.metrics.MetricRegistry@5c7579a4
[junit4] 2> 485249 INFO (coreCloseExecutor-1394-thread-1) [n:127.0.0.1:45667_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=626729946
[junit4] 2> 485250 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2a2ce3be{/solr,null,UNAVAILABLE}
[junit4] 2> 485329 INFO (zkCallback-897-thread-3-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 485329 INFO (zkCallback-887-thread-1-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 485329 INFO (zkCallback-897-thread-2-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 485329 INFO (zkCallback-887-thread-2-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 485447 INFO (OverseerCollectionConfigSetProcessor-99352316787556361-127.0.0.1:41443_solr-n_0000000000) [n:127.0.0.1:41443_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 485452 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[956EAD68A618714B]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testAddRemoveNode
[junit4] 2> 485461 INFO (TEST-TestClusterStateProvider.testAutoScalingConfig-seed#[956EAD68A618714B]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAutoScalingConfig
[junit4] 2> 485463 INFO (TEST-TestClusterStateProvider.testAutoScalingConfig-seed#[956EAD68A618714B]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testAutoScalingConfig
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestClusterStateProvider -Dtests.method=testAutoScalingConfig -Dtests.seed=956EAD68A618714B -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=fr-PF -Dtests.timezone=Etc/GMT-6 -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] FAILURE 0.01s J2 | TestClusterStateProvider.testAutoScalingConfig <<<
[junit4] > Throwable #1: java.lang.AssertionError: expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
[junit4] > "cluster-preferences":[{"maximize":"freedisk"}],
[junit4] > "triggers":{".auto_add_replicas":{
[junit4] > "name":".auto_add_replicas",
[junit4] > "event":"nodeLost",
[junit4] > "waitFor":30,
[junit4] > "actions":[
[junit4] > {
[junit4] > "name":"auto_add_replicas_plan",
[junit4] > "class":"solr.AutoAddReplicasPlanAction"},
[junit4] > {
[junit4] > "name":"execute_plan",
[junit4] > "class":"solr.ExecutePlanAction"}],
[junit4] > "enabled":true}},
[junit4] > "listeners":{".auto_add_replicas.system":{
[junit4] > "trigger":".auto_add_replicas",
[junit4] > "afterAction":[],
[junit4] > "stage":[
[junit4] > "STARTED",
[junit4] > "ABORTED",
[junit4] > "SUCCEEDED",
[junit4] > "FAILED",
[junit4] > "BEFORE_ACTION",
[junit4] > "AFTER_ACTION",
[junit4] > "IGNORED"],
[junit4] > "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
[junit4] > "beforeAction":[]}},
[junit4] > "properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
[junit4] > "cluster-preferences":[{"maximize":"freedisk"}],
[junit4] > "triggers":{".auto_add_replicas":{
[junit4] > "name":".auto_add_replicas",
[junit4] > "event":"nodeLost",
[junit4] > "waitFor":30,
[junit4] > "actions":[
[junit4] > {
[junit4] > "name":"auto_add_replicas_plan",
[junit4] > "class":"solr.AutoAddReplicasPlanAction"},
[junit4] > {
[junit4] > "name":"execute_plan",
[junit4] > "class":"solr.ExecutePlanAction"}],
[junit4] > "enabled":true}},
[junit4] > "listeners":{".auto_add_replicas.system":{
[junit4] > "trigger":".auto_add_replicas",
[junit4] > "afterAction":[],
[junit4] > "stage":[
[junit4] > "STARTED",
[junit4] > "ABORTED",
[junit4] > "SUCCEEDED",
[junit4] > "FAILED",
[junit4] > "BEFORE_ACTION",
[junit4] > "AFTER_ACTION",
[junit4] > "IGNORED"],
[junit4] > "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
[junit4] > "beforeAction":[]}},
[junit4] > "properties":{}}>
[junit4] > at __randomizedtesting.SeedInfo.seed([956EAD68A618714B:AAE6AEC0B14E81AC]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig(TestClusterStateProvider.java:214)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[junit4] > at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[junit4] > at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[junit4] > at java.base/java.lang.reflect.Method.invoke(Method.java:564)
[junit4] > at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> 485465 WARN (Simulated OverseerAutoScalingTriggerThread) [ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 485469 INFO (jetty-closer-870-thread-3) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@68ec6a00{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 485469 INFO (jetty-closer-870-thread-3) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=1049876059
[junit4] 2> 485469 INFO (jetty-closer-870-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 485469 INFO (jetty-closer-870-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7f0eb71e{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 485469 INFO (jetty-closer-870-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3a45b0d5: rootName = solr_37513, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1655e76a
[junit4] 2> 485469 INFO (jetty-closer-870-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2029198474
[junit4] 2> 485469 INFO (jetty-closer-870-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@779f1db0{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 485469 INFO (jetty-closer-870-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=987651807
[junit4] 2> 485469 INFO (jetty-closer-870-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 485469 INFO (jetty-closer-870-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7764375b: rootName = solr_41443, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@3ebb251f
[junit4] 2> 485470 INFO (jetty-closer-870-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 485470 INFO (jetty-closer-870-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7c4442d9: rootName = solr_46393, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@18f54a1a
[junit4] 2> 485473 INFO (jetty-closer-870-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 485473 INFO (jetty-closer-870-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@20100709: rootName = solr_41443, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7708928b
[junit4] 2> 485474 INFO (jetty-closer-870-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 485474 INFO (jetty-closer-870-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@fc95baf: rootName = solr_46393, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7708928b
[junit4] 2> 485475 INFO (jetty-closer-870-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 485475 INFO (jetty-closer-870-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@20edafc6: rootName = solr_37513, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7708928b
[junit4] 2> 485476 INFO (jetty-closer-870-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 485476 INFO (jetty-closer-870-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7ef0384c: rootName = solr_41443, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@453cff07
[junit4] 2> 485476 INFO (jetty-closer-870-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:41443_solr
[junit4] 2> 485477 INFO (jetty-closer-870-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 485477 INFO (jetty-closer-870-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 485477 INFO (jetty-closer-870-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5a7d18b2: rootName = solr_46393, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@453cff07
[junit4] 2> 485477 INFO (zkCallback-915-thread-1-processing-n:127.0.0.1:37513_solr) [n:127.0.0.1:37513_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 485477 INFO (zkCallback-897-thread-2-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 485477 INFO (zkCallback-887-thread-2-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (2)
[junit4] 2> 485477 INFO (coreCloseExecutor-1396-thread-1) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [.system_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@1577be7e
[junit4] 2> 485477 INFO (coreCloseExecutor-1396-thread-1) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core..system.shard1.replica_n1, tag=360169086
[junit4] 2> 485477 INFO (coreCloseExecutor-1396-thread-1) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6ebb85a8: rootName = solr_41443, domain = solr.core..system.shard1.replica_n1, service url = null, agent id = null] for registry solr.core..system.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@7327fcc3
[junit4] 2> 485478 INFO (jetty-closer-870-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:46393_solr
[junit4] 2> 485478 INFO (zkCallback-915-thread-1-processing-n:127.0.0.1:37513_solr) [n:127.0.0.1:37513_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 485478 INFO (zkCallback-897-thread-2-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 485478 INFO (jetty-closer-870-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 485478 INFO (zkCallback-887-thread-2-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 485479 INFO (coreCloseExecutor-1397-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [.system_shard1_replica_p4] CLOSING SolrCore org.apache.solr.core.SolrCore@4a6cb6d9
[junit4] 2> 485479 INFO (coreCloseExecutor-1397-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core..system.shard1.replica_p4, tag=1248638681
[junit4] 2> 485479 INFO (coreCloseExecutor-1397-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@fc030b0: rootName = solr_46393, domain = solr.core..system.shard1.replica_p4, service url = null, agent id = null] for registry solr.core..system.shard1.replica_p4 / com.codahale.metrics.MetricRegistry@1b333cd6
[junit4] 2> 485480 INFO (jetty-closer-870-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 485480 INFO (jetty-closer-870-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@242bf00e: rootName = solr_37513, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@453cff07
[junit4] 2> 485480 INFO (jetty-closer-870-thread-3) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37513_solr
[junit4] 2> 485481 INFO (jetty-closer-870-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 485481 INFO (zkCallback-897-thread-2-processing-n:127.0.0.1:41443_solr) [n:127.0.0.1:41443_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 485481 INFO (zkCallback-915-thread-1-processing-n:127.0.0.1:37513_solr) [n:127.0.0.1:37513_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 485481 INFO (jetty-closer-870-thread-3) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 485482 INFO (jetty-closer-870-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@757e1e98{/solr,null,UNAVAILABLE}
[junit4] 2> 485486 INFO (coreCloseExecutor-1396-thread-1) [n:127.0.0.1:41443_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=360169086
[junit4] 2> 485487 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f867ddbc000e, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> 485488 INFO (coreCloseExecutor-1397-thread-1) [n:127.0.0.1:46393_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=1248638681
[junit4] 2> 485488 INFO (zkCallback-887-thread-2-processing-n:127.0.0.1:46393_solr) [n:127.0.0.1:46393_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 485492 INFO (jetty-closer-870-thread-2) [ ] o.a.s.c.Overseer Overseer (id=99352316787556361-127.0.0.1:41443_solr-n_0000000000) closing
[junit4] 2> 485492 INFO (OverseerStateUpdate-99352316787556361-127.0.0.1:41443_solr-n_0000000000) [n:127.0.0.1:41443_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41443_solr
[junit4] 2> 485492 WARN (OverseerAutoScalingTriggerThread-99352316787556361-127.0.0.1:41443_solr-n_0000000000) [n:127.0.0.1:41443_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 485494 INFO (jetty-closer-870-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@5b954d4f{/solr,null,UNAVAILABLE}
[junit4] 2> 485495 INFO (jetty-closer-870-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@26b21342{/solr,null,UNAVAILABLE}
[junit4] 2> 485496 ERROR (SUITE-TestClusterStateProvider-seed#[956EAD68A618714B]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 485496 INFO (SUITE-TestClusterStateProvider-seed#[956EAD68A618714B]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35371 35371
[junit4] 2> 491278 INFO (Thread-515) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35371 35371
[junit4] 2> 491279 WARN (Thread-515) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 3 /solr/configs/.system
[junit4] 2> 3 /solr/configs/.system/managed-schema
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 20 /solr/collections/.system/state.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 2 /solr/overseer_elect/election/99352316787556359-127.0.0.1:46393_solr-n_0000000001
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_956EAD68A618714B-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@31c3e760), locale=fr-PF, timezone=Etc/GMT-6
[junit4] 2> NOTE: Linux 4.13.0-26-generic amd64/Oracle Corporation 9.0.1 (64-bit)/cpus=8,threads=1,free=136544664,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [TestSerializedLuceneMatchVersion, TestNumericTerms64, UniqFieldsUpdateProcessorFactoryTest, IndexBasedSpellCheckerTest, TestTolerantSearch, ZkFailoverTest, TestCSVResponseWriter, CdcrBidirectionalTest, UnloadDistributedZkTest, TestExpandComponent, SystemLogListenerTest, TestCharFilters, TestInfoStreamLogging, TestExceedMaxTermLength, PeerSyncWithIndexFingerprintCachingTest, ConfigureRecoveryStrategyTest, TestCoreDiscovery, LeaderFailureAfterFreshStartTest, LeaderElectionTest, SpellPossibilityIteratorTest, TestValueSourceCache, TestCoreContainer, SolrGraphiteReporterTest, BigEndianAscendingWordSerializerTest, TestExtendedDismaxParser, TimeZoneUtilsTest, SolrTestCaseJ4Test, RequiredFieldsTest, TestApiFramework, TestQueryTypes, ReturnFieldsTest, TestCloudNestedDocsSort, TestSolrCloudWithKerberosAlt, SolrCLIZkUtilsTest, ConnectionReuseTest, BadComponentTest, TestManagedSchemaThreadSafety, CopyFieldTest, TestCollapseQParserPlugin, NodeAddedTriggerTest, UtilsToolTest, TestGeoJSONResponseWriter, NumberUtilsTest, TestHashPartitioner, TestLegacyFieldReuse, XmlUpdateRequestHandlerTest, BlobRepositoryCloudTest, TestPKIAuthenticationPlugin, TestClusterStateProvider]
[junit4] Completed [145/768 (1!)] on J2 in 10.19s, 2 tests, 1 failure <<< FAILURES!
[...truncated 42948 lines...]
[JENKINS] Lucene-Solr-master-Linux (32bit/jdk1.8.0_144) - Build #
21275 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Linux/21275/
Java: 32bit/jdk1.8.0_144 -client -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig
Error Message:
expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{ "cluster-preferences":[{"maximize":"freedisk"}], "triggers":{".auto_add_replicas":{ "name":".auto_add_replicas", "event":"nodeLost", "waitFor":30, "actions":[ { "name":"auto_add_replicas_plan", "class":"solr.AutoAddReplicasPlanAction"}, { "name":"execute_plan", "class":"solr.ExecutePlanAction"}], "enabled":true}}, "listeners":{".auto_add_replicas.system":{ "trigger":".auto_add_replicas", "afterAction":[], "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED", "BEFORE_ACTION", "AFTER_ACTION", "IGNORED"], "class":"org.apache.solr.cloud.autoscaling.SystemLogListener", "beforeAction":[]}}, "properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{ "cluster-preferences":[{"maximize":"freedisk"}], "triggers":{".auto_add_replicas":{ "name":".auto_add_replicas", "event":"nodeLost", "waitFor":30, "actions":[ { "name":"auto_add_replicas_plan", "class":"solr.AutoAddReplicasPlanAction"}, { "name":"execute_plan", "class":"solr.ExecutePlanAction"}], "enabled":true}}, "listeners":{".auto_add_replicas.system":{ "trigger":".auto_add_replicas", "afterAction":[], "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED", "BEFORE_ACTION", "AFTER_ACTION", "IGNORED"], "class":"org.apache.solr.cloud.autoscaling.SystemLogListener", "beforeAction":[]}}, "properties":{}}>
Stack Trace:
java.lang.AssertionError: expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
"cluster-preferences":[{"maximize":"freedisk"}],
"triggers":{".auto_add_replicas":{
"name":".auto_add_replicas",
"event":"nodeLost",
"waitFor":30,
"actions":[
{
"name":"auto_add_replicas_plan",
"class":"solr.AutoAddReplicasPlanAction"},
{
"name":"execute_plan",
"class":"solr.ExecutePlanAction"}],
"enabled":true}},
"listeners":{".auto_add_replicas.system":{
"trigger":".auto_add_replicas",
"afterAction":[],
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED",
"BEFORE_ACTION",
"AFTER_ACTION",
"IGNORED"],
"class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
"beforeAction":[]}},
"properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
"cluster-preferences":[{"maximize":"freedisk"}],
"triggers":{".auto_add_replicas":{
"name":".auto_add_replicas",
"event":"nodeLost",
"waitFor":30,
"actions":[
{
"name":"auto_add_replicas_plan",
"class":"solr.AutoAddReplicasPlanAction"},
{
"name":"execute_plan",
"class":"solr.ExecutePlanAction"}],
"enabled":true}},
"listeners":{".auto_add_replicas.system":{
"trigger":".auto_add_replicas",
"afterAction":[],
"stage":[
"STARTED",
"ABORTED",
"SUCCEEDED",
"FAILED",
"BEFORE_ACTION",
"AFTER_ACTION",
"IGNORED"],
"class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
"beforeAction":[]}},
"properties":{}}>
at __randomizedtesting.SeedInfo.seed([AD558E4DD12B51C3:92DD8DE5C67DA124]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:147)
at org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig(TestClusterStateProvider.java:214)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13664 lines...]
[junit4] Suite: org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/init-core-data-001
[junit4] 2> 3570083 WARN (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=30 numCloses=30
[junit4] 2> 3570083 INFO (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 3570085 INFO (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 3570086 INFO (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 3570086 INFO (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.s.c.a.s.TestClusterStateProvider ####### Using simulated components? false
[junit4] 2> 3570087 INFO (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/tempDir-001
[junit4] 2> 3570087 INFO (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 3570087 INFO (Thread-4580) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 3570087 INFO (Thread-4580) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 3570089 ERROR (Thread-4580) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3570187 INFO (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46433
[junit4] 2> 3570191 INFO (zkConnectionManagerCallback-3558-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570201 INFO (jetty-launcher-3555-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 3570201 INFO (jetty-launcher-3555-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 3570201 INFO (jetty-launcher-3555-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 3570204 INFO (jetty-launcher-3555-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@15ee55e{/solr,null,AVAILABLE}
[junit4] 2> 3570204 INFO (jetty-launcher-3555-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@186ce2a{/solr,null,AVAILABLE}
[junit4] 2> 3570204 INFO (jetty-launcher-3555-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1190e3b{/solr,null,AVAILABLE}
[junit4] 2> 3570210 INFO (jetty-launcher-3555-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@a20e53{SSL,[ssl, http/1.1]}{127.0.0.1:45153}
[junit4] 2> 3570210 INFO (jetty-launcher-3555-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1d642f7{SSL,[ssl, http/1.1]}{127.0.0.1:39865}
[junit4] 2> 3570210 INFO (jetty-launcher-3555-thread-2) [ ] o.e.j.s.Server Started @3572171ms
[junit4] 2> 3570210 INFO (jetty-launcher-3555-thread-1) [ ] o.e.j.s.Server Started @3572171ms
[junit4] 2> 3570210 INFO (jetty-launcher-3555-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45153}
[junit4] 2> 3570210 INFO (jetty-launcher-3555-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39865}
[junit4] 2> 3570210 INFO (jetty-launcher-3555-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@18105a1{SSL,[ssl, http/1.1]}{127.0.0.1:43773}
[junit4] 2> 3570211 INFO (jetty-launcher-3555-thread-3) [ ] o.e.j.s.Server Started @3572171ms
[junit4] 2> 3570211 INFO (jetty-launcher-3555-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43773}
[junit4] 2> 3570211 ERROR (jetty-launcher-3555-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3570211 ERROR (jetty-launcher-3555-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3570211 INFO (jetty-launcher-3555-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 3570211 ERROR (jetty-launcher-3555-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3570211 INFO (jetty-launcher-3555-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3570211 INFO (jetty-launcher-3555-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 3570211 INFO (jetty-launcher-3555-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3570212 INFO (jetty-launcher-3555-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3570212 INFO (jetty-launcher-3555-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T01:57:22.141Z
[junit4] 2> 3570212 INFO (jetty-launcher-3555-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3570211 INFO (jetty-launcher-3555-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 3570212 INFO (jetty-launcher-3555-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T01:57:22.141Z
[junit4] 2> 3570212 INFO (jetty-launcher-3555-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3570212 INFO (jetty-launcher-3555-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3570212 INFO (jetty-launcher-3555-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T01:57:22.141Z
[junit4] 2> 3570214 INFO (zkConnectionManagerCallback-3560-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570215 INFO (zkConnectionManagerCallback-3564-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570215 INFO (jetty-launcher-3555-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3570216 INFO (zkConnectionManagerCallback-3562-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570216 INFO (jetty-launcher-3555-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3570216 INFO (jetty-launcher-3555-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3570267 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f7890be10002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3570270 INFO (jetty-launcher-3555-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46433/solr
[junit4] 2> 3570270 INFO (jetty-launcher-3555-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46433/solr
[junit4] 2> 3570270 INFO (jetty-launcher-3555-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46433/solr
[junit4] 2> 3570271 INFO (zkConnectionManagerCallback-3572-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570272 INFO (zkConnectionManagerCallback-3576-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570272 INFO (zkConnectionManagerCallback-3574-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570273 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f7890be10004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3570273 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f7890be10006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3570275 INFO (zkConnectionManagerCallback-3578-thread-1-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570275 INFO (zkConnectionManagerCallback-3580-thread-1-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570275 INFO (zkConnectionManagerCallback-3582-thread-1-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570322 INFO (jetty-launcher-3555-thread-3) [n:127.0.0.1:43773_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3570322 INFO (jetty-launcher-3555-thread-2) [n:127.0.0.1:45153_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3570322 INFO (jetty-launcher-3555-thread-1) [n:127.0.0.1:39865_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3570322 INFO (jetty-launcher-3555-thread-3) [n:127.0.0.1:43773_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:43773_solr
[junit4] 2> 3570322 INFO (jetty-launcher-3555-thread-2) [n:127.0.0.1:45153_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45153_solr
[junit4] 2> 3570322 INFO (jetty-launcher-3555-thread-1) [n:127.0.0.1:39865_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39865_solr
[junit4] 2> 3570322 INFO (jetty-launcher-3555-thread-3) [n:127.0.0.1:43773_solr ] o.a.s.c.Overseer Overseer (id=99351359784026121-127.0.0.1:43773_solr-n_0000000000) starting
[junit4] 2> 3570327 INFO (zkCallback-3579-thread-1-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3570327 INFO (zkCallback-3581-thread-1-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3570328 INFO (zkCallback-3577-thread-1-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 3570339 INFO (jetty-launcher-3555-thread-3) [n:127.0.0.1:43773_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43773_solr
[junit4] 2> 3570340 INFO (zkCallback-3579-thread-1-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3570340 INFO (zkCallback-3581-thread-1-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3570340 INFO (zkCallback-3577-thread-1-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 3570484 INFO (jetty-launcher-3555-thread-2) [n:127.0.0.1:45153_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45153.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3570487 INFO (jetty-launcher-3555-thread-1) [n:127.0.0.1:39865_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39865.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3570496 INFO (jetty-launcher-3555-thread-2) [n:127.0.0.1:45153_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45153.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3570497 INFO (jetty-launcher-3555-thread-2) [n:127.0.0.1:45153_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45153.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3570498 INFO (jetty-launcher-3555-thread-2) [n:127.0.0.1:45153_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/tempDir-001/node2/.
[junit4] 2> 3570500 INFO (jetty-launcher-3555-thread-1) [n:127.0.0.1:39865_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39865.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3570501 INFO (jetty-launcher-3555-thread-1) [n:127.0.0.1:39865_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39865.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3570503 INFO (jetty-launcher-3555-thread-1) [n:127.0.0.1:39865_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/tempDir-001/node1/.
[junit4] 2> 3570563 INFO (jetty-launcher-3555-thread-3) [n:127.0.0.1:43773_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43773.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3570576 INFO (jetty-launcher-3555-thread-3) [n:127.0.0.1:43773_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43773.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3570576 INFO (jetty-launcher-3555-thread-3) [n:127.0.0.1:43773_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43773.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3570577 INFO (jetty-launcher-3555-thread-3) [n:127.0.0.1:43773_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/tempDir-001/node3/.
[junit4] 2> 3570612 INFO (zkConnectionManagerCallback-3590-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570615 INFO (zkConnectionManagerCallback-3594-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3570616 INFO (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3570616 INFO (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46433/solr ready
[junit4] 2> 3570678 INFO (qtp2763473-17825) [n:127.0.0.1:39865_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=1&replicationFactor=2&name=.system&nrtReplicas=2&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 3570683 INFO (OverseerThreadFactory-6693-thread-1-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.CreateCollectionCmd Create collection .system
[junit4] 2> 3570790 INFO (OverseerStateUpdate-99351359784026121-127.0.0.1:43773_solr-n_0000000000) [n:127.0.0.1:43773_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:45153/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3570793 INFO (OverseerStateUpdate-99351359784026121-127.0.0.1:43773_solr-n_0000000000) [n:127.0.0.1:43773_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:39865/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3570795 INFO (OverseerStateUpdate-99351359784026121-127.0.0.1:43773_solr-n_0000000000) [n:127.0.0.1:43773_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":".system",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":".system_shard1_replica_p4",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43773/solr",
[junit4] 2> "type":"PULL",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 3571005 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n2&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3571006 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3571084 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n1&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 3571084 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=.system&newCollection=true&name=.system_shard1_replica_p4&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=PULL
[junit4] 2> 3571084 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3571084 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3571189 INFO (zkCallback-3581-thread-1-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3571189 INFO (zkCallback-3577-thread-1-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3571189 INFO (zkCallback-3579-thread-1-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3571190 INFO (zkCallback-3579-thread-2-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3571190 INFO (zkCallback-3581-thread-2-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3571190 INFO (zkCallback-3577-thread-2-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3572033 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 3572044 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 3572050 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.IndexSchema [.system_shard1_replica_n2] Schema name=_system collection or core
[junit4] 2> 3572057 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 3572058 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/.system/managed-schema
[junit4] 2> 3572059 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/.system/schema.xml to /configs/.system/schema.xml.bak
[junit4] 2> 3572059 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_n2' using configuration from collection .system, trusted=true
[junit4] 2> 3572060 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39865.solr.core..system.shard1.replica_n2' (registry 'solr.core..system.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3572061 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3572061 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore [[.system_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/tempDir-001/node1/.system_shard1_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/tempDir-001/node1/./.system_shard1_replica_n2/data/]
[junit4] 2> 3572104 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 3572104 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 3572113 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.IndexSchema [.system_shard1_replica_n1] Schema name=_system collection or core
[junit4] 2> 3572114 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.IndexSchema [.system_shard1_replica_p4] Schema name=_system collection or core
[junit4] 2> 3572118 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 3572118 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_n1' using configuration from collection .system, trusted=true
[junit4] 2> 3572119 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45153.solr.core..system.shard1.replica_n1' (registry 'solr.core..system.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3572119 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3572119 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [[.system_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/tempDir-001/node2/.system_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/tempDir-001/node2/./.system_shard1_replica_n1/data/]
[junit4] 2> 3572120 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.IndexSchema Loaded schema _system collection or core/1.1 with uniqueid field id
[junit4] 2> 3572120 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.CoreContainer Creating SolrCore '.system_shard1_replica_p4' using configuration from collection .system, trusted=true
[junit4] 2> 3572121 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43773.solr.core..system.shard1.replica_p4' (registry 'solr.core..system.shard1.replica_p4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3572121 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 3572121 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [[.system_shard1_replica_p4] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/tempDir-001/node3/.system_shard1_replica_p4], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/tempDir-001/node3/./.system_shard1_replica_p4/data/]
[junit4] 2> 3572125 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3572125 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3572127 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: if 1 uncommited docs
[junit4] 2> 3572127 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3572128 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@18a6531[.system_shard1_replica_n2] main]
[junit4] 2> 3572129 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/.system
[junit4] 2> 3572130 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/.system
[junit4] 2> 3572130 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/.system/managed-schema
[junit4] 2> 3572130 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 3572131 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3572132 INFO (searcherExecutor-6700-thread-1-processing-n:127.0.0.1:39865_solr x:.system_shard1_replica_n2 s:shard1 c:.system r:core_node5) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore [.system_shard1_replica_n2] Registered new searcher Searcher@18a6531[.system_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3572132 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589621758687707136
[junit4] 2> 3572136 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 3572171 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.CommitTracker Hard AutoCommit: if 1 uncommited docs
[junit4] 2> 3572171 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3572173 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 3572173 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 3572173 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.SolrIndexSearcher Opening [Searcher@eff1d6[.system_shard1_replica_p4] main]
[junit4] 2> 3572174 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/.system
[junit4] 2> 3572174 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: if 1 uncommited docs
[junit4] 2> 3572174 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 3572174 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/.system
[junit4] 2> 3572175 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/.system/managed-schema
[junit4] 2> 3572175 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 3572176 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3572176 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ac790f[.system_shard1_replica_n1] main]
[junit4] 2> 3572177 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.ZkController .system_shard1_replica_p4 starting background replication from leader
[junit4] 2> 3572177 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 3572177 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/.system
[junit4] 2> 3572177 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/.system
[junit4] 2> 3572177 INFO (searcherExecutor-6703-thread-1-processing-n:127.0.0.1:43773_solr x:.system_shard1_replica_p4 s:shard1 c:.system r:core_node6) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [.system_shard1_replica_p4] Registered new searcher Searcher@eff1d6[.system_shard1_replica_p4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3572177 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/.system/managed-schema
[junit4] 2> 3572178 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 3572178 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 3572178 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3572179 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 3572180 INFO (searcherExecutor-6702-thread-1-processing-n:127.0.0.1:45153_solr x:.system_shard1_replica_n1 s:shard1 c:.system r:core_node3) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [.system_shard1_replica_n1] Registered new searcher Searcher@1ac790f[.system_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3572180 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589621758738038784
[junit4] 2> 3572238 INFO (zkCallback-3579-thread-2-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3572238 INFO (zkCallback-3577-thread-1-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3572238 INFO (zkCallback-3577-thread-2-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3572238 INFO (zkCallback-3579-thread-1-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3572238 INFO (zkCallback-3581-thread-2-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3572238 INFO (zkCallback-3581-thread-1-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3572637 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 3572637 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 3572637 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:39865/solr/.system_shard1_replica_n2/
[junit4] 2> 3572638 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=.system_shard1_replica_n2 url=https://127.0.0.1:39865/solr START replicas=[https://127.0.0.1:45153/solr/.system_shard1_replica_n1/] nUpdates=100
[junit4] 2> 3572642 INFO (qtp22616256-17832) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.S.Request [.system_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 3572938 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=.system_shard1_replica_n2 url=https://127.0.0.1:39865/solr DONE. We have no versions. sync failed.
[junit4] 2> 3572938 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 3572938 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 3572938 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 3572944 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:39865/solr/.system_shard1_replica_n2/ shard1
[junit4] 2> 3573046 INFO (zkCallback-3577-thread-2-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3573046 INFO (zkCallback-3581-thread-2-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3573046 INFO (zkCallback-3577-thread-1-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3573046 INFO (zkCallback-3581-thread-1-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3573046 INFO (zkCallback-3579-thread-1-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3573046 INFO (zkCallback-3579-thread-2-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3573080 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Replica core_node5 is leader but it's state is down, skipping replication
[junit4] 2> 3573095 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 3573121 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n2&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2116
[junit4] 2> 3573183 INFO (qtp14806180-17823) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=.system&newCollection=true&name=.system_shard1_replica_p4&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=PULL} status=0 QTime=2099
[junit4] 2> 3573191 INFO (qtp22616256-17829) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=.system&newCollection=true&name=.system_shard1_replica_n1&action=CREATE&numShards=1&collection=.system&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2107
[junit4] 2> 3573198 INFO (qtp2763473-17825) [n:127.0.0.1:39865_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 3573288 INFO (zkCallback-3581-thread-1-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3573288 INFO (zkCallback-3577-thread-1-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3573288 INFO (zkCallback-3579-thread-1-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3573288 INFO (zkCallback-3577-thread-2-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3573288 INFO (zkCallback-3579-thread-2-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3573288 INFO (zkCallback-3581-thread-2-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3574199 INFO (qtp2763473-17825) [n:127.0.0.1:39865_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=1&replicationFactor=2&name=.system&nrtReplicas=2&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=3520
[junit4] 2> 3574204 INFO (qtp2763473-17825) [n:127.0.0.1:39865_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 3574209 INFO (qtp14806180-17836) [n:127.0.0.1:43773_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 3574212 INFO (qtp22616256-17832) [n:127.0.0.1:45153_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
[junit4] 2> 3574236 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAddRemoveNode
[junit4] 2> 3574238 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 3574239 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1d28ce1{/solr,null,AVAILABLE}
[junit4] 2> 3574240 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@111b056{SSL,[ssl, http/1.1]}{127.0.0.1:39905}
[junit4] 2> 3574240 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.e.j.s.Server Started @3576200ms
[junit4] 2> 3574240 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=39905}
[junit4] 2> 3574240 ERROR (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 3574241 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 3574241 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 3574241 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 3574241 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-15T01:57:26.170Z
[junit4] 2> 3574242 INFO (zkConnectionManagerCallback-3596-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3574243 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 3574250 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46433/solr
[junit4] 2> 3574252 INFO (zkConnectionManagerCallback-3600-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3574254 INFO (zkConnectionManagerCallback-3602-thread-1-processing-n:127.0.0.1:39905_solr) [n:127.0.0.1:39905_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
[junit4] 2> 3574257 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [n:127.0.0.1:39905_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 3574258 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [n:127.0.0.1:39905_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 3574259 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [n:127.0.0.1:39905_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:39905_solr
[junit4] 2> 3574259 INFO (zkCallback-3593-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3574259 INFO (zkCallback-3579-thread-2-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3574259 INFO (zkCallback-3581-thread-2-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3574260 INFO (zkCallback-3601-thread-1-processing-n:127.0.0.1:39905_solr) [n:127.0.0.1:39905_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3574259 INFO (zkCallback-3577-thread-2-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 3574503 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [n:127.0.0.1:39905_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39905.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3574514 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [n:127.0.0.1:39905_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39905.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3574514 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [n:127.0.0.1:39905_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_39905.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@8e57a9
[junit4] 2> 3574516 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [n:127.0.0.1:39905_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001/tempDir-001/node4/.
[junit4] 2> 3574686 INFO (OverseerCollectionConfigSetProcessor-99351359784026121-127.0.0.1:43773_solr-n_0000000000) [n:127.0.0.1:43773_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 3575178 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication
[junit4] 2> 3575179 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Updated masterUrl to https://127.0.0.1:39865/solr/.system_shard1_replica_n2/
[junit4] 2> 3575183 INFO (qtp2763473-17831) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.S.Request [.system_shard1_replica_n2] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 3575184 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 3575184 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 3575184 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 3575185 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 3575185 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 3575185 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3575185 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@b63c3d commitCommandVersion:0
[junit4] 2> 3575187 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.SolrIndexSearcher Opening [Searcher@9d1690[.system_shard1_replica_p4] main]
[junit4] 2> 3575188 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3575192 INFO (searcherExecutor-6703-thread-1-processing-n:127.0.0.1:43773_solr x:.system_shard1_replica_p4 s:shard1 c:.system r:core_node6) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [.system_shard1_replica_p4] Registered new searcher Searcher@9d1690[.system_shard1_replica_p4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3576566 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@a20e53{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 3576566 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=8602097
[junit4] 2> 3576566 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 3576567 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@35875b: rootName = solr_45153, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1aee49f
[junit4] 2> 3576573 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 3576573 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@6e4453: rootName = solr_45153, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@17396ad
[junit4] 2> 3576579 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 3576579 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1bdce13: rootName = solr_45153, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@114bba7
[junit4] 2> 3576581 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45153_solr
[junit4] 2> 3576582 INFO (zkCallback-3593-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 3576582 INFO (zkCallback-3601-thread-1-processing-n:127.0.0.1:39905_solr) [n:127.0.0.1:39905_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 3576582 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 3576582 INFO (zkCallback-3579-thread-2-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 3576582 INFO (zkCallback-3581-thread-2-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 3576582 INFO (zkCallback-3577-thread-2-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (3)
[junit4] 2> 3576583 INFO (zkCallback-3577-thread-1-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3576583 INFO (zkCallback-3577-thread-3-processing-n:127.0.0.1:45153_solr) [n:127.0.0.1:45153_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3576583 INFO (zkCallback-3581-thread-1-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3576583 INFO (zkCallback-3581-thread-3-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3576583 INFO (zkCallback-3579-thread-1-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3576584 INFO (zkCallback-3579-thread-3-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [3])
[junit4] 2> 3576584 INFO (coreCloseExecutor-6722-thread-1) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [.system_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@10bfe2
[junit4] 2> 3576584 INFO (coreCloseExecutor-6722-thread-1) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core..system.shard1.replica_n1, tag=1097698
[junit4] 2> 3576584 INFO (coreCloseExecutor-6722-thread-1) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1bf3e71: rootName = solr_45153, domain = solr.core..system.shard1.replica_n1, service url = null, agent id = null] for registry solr.core..system.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@3f883e
[junit4] 2> 3576594 INFO (coreCloseExecutor-6722-thread-1) [n:127.0.0.1:45153_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=1097698
[junit4] 2> 3576596 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@186ce2a{/solr,null,UNAVAILABLE}
[junit4] 2> 3578180 INFO (qtp2763473-17825) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.S.Request [.system_shard1_replica_n2] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 3578181 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 3578181 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 3578181 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Slave's generation: 2
[junit4] 2> 3578181 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher Slave's version: 1515981447114
[junit4] 2> 3578181 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 3578181 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 3578181 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@b63c3d commitCommandVersion:0
[junit4] 2> 3578182 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.s.SolrIndexSearcher Opening [Searcher@1d4c4af[.system_shard1_replica_p4] main]
[junit4] 2> 3578182 INFO (indexFetcher-6715-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 3578186 INFO (searcherExecutor-6703-thread-1-processing-n:127.0.0.1:43773_solr x:.system_shard1_replica_p4 s:shard1 c:.system r:core_node6) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.c.SolrCore [.system_shard1_replica_p4] Registered new searcher Searcher@1d4c4af[.system_shard1_replica_p4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 3578597 INFO (TEST-TestClusterStateProvider.testAddRemoveNode-seed#[AD558E4DD12B51C3]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testAddRemoveNode
[junit4] 2> 3578629 INFO (TEST-TestClusterStateProvider.testAutoScalingConfig-seed#[AD558E4DD12B51C3]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testAutoScalingConfig
[junit4] 2> 3578632 INFO (TEST-TestClusterStateProvider.testAutoScalingConfig-seed#[AD558E4DD12B51C3]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testAutoScalingConfig
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestClusterStateProvider -Dtests.method=testAutoScalingConfig -Dtests.seed=AD558E4DD12B51C3 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=sk-SK -Dtests.timezone=Etc/GMT+6 -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] FAILURE 0.04s J2 | TestClusterStateProvider.testAutoScalingConfig <<<
[junit4] > Throwable #1: java.lang.AssertionError: expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
[junit4] > "cluster-preferences":[{"maximize":"freedisk"}],
[junit4] > "triggers":{".auto_add_replicas":{
[junit4] > "name":".auto_add_replicas",
[junit4] > "event":"nodeLost",
[junit4] > "waitFor":30,
[junit4] > "actions":[
[junit4] > {
[junit4] > "name":"auto_add_replicas_plan",
[junit4] > "class":"solr.AutoAddReplicasPlanAction"},
[junit4] > {
[junit4] > "name":"execute_plan",
[junit4] > "class":"solr.ExecutePlanAction"}],
[junit4] > "enabled":true}},
[junit4] > "listeners":{".auto_add_replicas.system":{
[junit4] > "trigger":".auto_add_replicas",
[junit4] > "afterAction":[],
[junit4] > "stage":[
[junit4] > "STARTED",
[junit4] > "ABORTED",
[junit4] > "SUCCEEDED",
[junit4] > "FAILED",
[junit4] > "BEFORE_ACTION",
[junit4] > "AFTER_ACTION",
[junit4] > "IGNORED"],
[junit4] > "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
[junit4] > "beforeAction":[]}},
[junit4] > "properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
[junit4] > "cluster-preferences":[{"maximize":"freedisk"}],
[junit4] > "triggers":{".auto_add_replicas":{
[junit4] > "name":".auto_add_replicas",
[junit4] > "event":"nodeLost",
[junit4] > "waitFor":30,
[junit4] > "actions":[
[junit4] > {
[junit4] > "name":"auto_add_replicas_plan",
[junit4] > "class":"solr.AutoAddReplicasPlanAction"},
[junit4] > {
[junit4] > "name":"execute_plan",
[junit4] > "class":"solr.ExecutePlanAction"}],
[junit4] > "enabled":true}},
[junit4] > "listeners":{".auto_add_replicas.system":{
[junit4] > "trigger":".auto_add_replicas",
[junit4] > "afterAction":[],
[junit4] > "stage":[
[junit4] > "STARTED",
[junit4] > "ABORTED",
[junit4] > "SUCCEEDED",
[junit4] > "FAILED",
[junit4] > "BEFORE_ACTION",
[junit4] > "AFTER_ACTION",
[junit4] > "IGNORED"],
[junit4] > "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
[junit4] > "beforeAction":[]}},
[junit4] > "properties":{}}>
[junit4] > at __randomizedtesting.SeedInfo.seed([AD558E4DD12B51C3:92DD8DE5C67DA124]:0)
[junit4] > at org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig(TestClusterStateProvider.java:214)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3578640 INFO (jetty-closer-3556-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@18105a1{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 3578640 INFO (jetty-closer-3556-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1d642f7{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 3578640 INFO (jetty-closer-3556-thread-3) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@111b056{SSL,[ssl, http/1.1]}{127.0.0.1:0}
[junit4] 2> 3578641 INFO (jetty-closer-3556-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=14330550
[junit4] 2> 3578641 INFO (jetty-closer-3556-thread-3) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=33407168
[junit4] 2> 3578641 INFO (jetty-closer-3556-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=20140885
[junit4] 2> 3578641 INFO (jetty-closer-3556-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 3578641 INFO (jetty-closer-3556-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 3578642 INFO (jetty-closer-3556-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 3578642 INFO (jetty-closer-3556-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@12b320e: rootName = solr_39905, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@a86c3a
[junit4] 2> 3578642 INFO (jetty-closer-3556-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@11c6259: rootName = solr_39865, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@7058b3
[junit4] 2> 3578642 INFO (jetty-closer-3556-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@163067e: rootName = solr_43773, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@527897
[junit4] 2> 3578652 INFO (jetty-closer-3556-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 3578652 INFO (jetty-closer-3556-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3d2c73: rootName = solr_39905, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@17396ad
[junit4] 2> 3578654 INFO (jetty-closer-3556-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 3578654 INFO (jetty-closer-3556-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@298f9a: rootName = solr_39865, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@17396ad
[junit4] 2> 3578654 INFO (jetty-closer-3556-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 3578654 INFO (jetty-closer-3556-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@d3792b: rootName = solr_43773, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@17396ad
[junit4] 2> 3578660 INFO (jetty-closer-3556-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 3578660 INFO (jetty-closer-3556-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@96990f: rootName = solr_39865, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@114bba7
[junit4] 2> 3578660 INFO (jetty-closer-3556-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 3578660 INFO (jetty-closer-3556-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@9cd3f5: rootName = solr_39905, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@114bba7
[junit4] 2> 3578661 INFO (jetty-closer-3556-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 3578661 INFO (jetty-closer-3556-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@133fda8: rootName = solr_43773, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@114bba7
[junit4] 2> 3578662 INFO (jetty-closer-3556-thread-3) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39905_solr
[junit4] 2> 3578662 INFO (jetty-closer-3556-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:39865_solr
[junit4] 2> 3578662 INFO (jetty-closer-3556-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 3578662 INFO (jetty-closer-3556-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 3578662 INFO (zkCallback-3601-thread-1-processing-n:127.0.0.1:39905_solr) [n:127.0.0.1:39905_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1)
[junit4] 2> 3578662 INFO (jetty-closer-3556-thread-3) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 3578663 INFO (zkCallback-3579-thread-3-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1)
[junit4] 2> 3578663 INFO (zkCallback-3581-thread-3-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (1)
[junit4] 2> 3578663 INFO (coreCloseExecutor-6724-thread-1) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore [.system_shard1_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@18d28c
[junit4] 2> 3578663 INFO (coreCloseExecutor-6724-thread-1) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core..system.shard1.replica_n2, tag=1626764
[junit4] 2> 3578663 INFO (jetty-closer-3556-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:43773_solr
[junit4] 2> 3578663 INFO (coreCloseExecutor-6724-thread-1) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@bf512b: rootName = solr_39865, domain = solr.core..system.shard1.replica_n2, service url = null, agent id = null] for registry solr.core..system.shard1.replica_n2 / com.codahale.metrics.MetricRegistry@8450c7
[junit4] 2> 3578664 INFO (zkCallback-3579-thread-1-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3578664 INFO (zkCallback-3581-thread-1-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3578664 INFO (zkCallback-3581-thread-3-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 3578664 INFO (zkCallback-3581-thread-2-processing-n:127.0.0.1:43773_solr) [n:127.0.0.1:43773_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [0])
[junit4] 2> 3578664 INFO (zkCallback-3579-thread-2-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/.system/state.json] for collection [.system] has occurred - updating... (live nodes size: [1])
[junit4] 2> 3578664 INFO (zkCallback-3579-thread-3-processing-n:127.0.0.1:39865_solr) [n:127.0.0.1:39865_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 3578664 INFO (jetty-closer-3556-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric re
[...truncated too long message...]
lrJmxReporter@1e6df4a: rootName = solr_43773, domain = solr.core..system.shard1.replica_p4, service url = null, agent id = null] for registry solr.core..system.shard1.replica_p4 / com.codahale.metrics.MetricRegistry@1f0e982
[junit4] 2> 3578674 INFO (coreCloseExecutor-6724-thread-1) [n:127.0.0.1:39865_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=1626764
[junit4] 2> 3578676 INFO (coreCloseExecutor-6726-thread-1) [n:127.0.0.1:43773_solr c:.system s:shard1 r:core_node6 x:.system_shard1_replica_p4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=13870298
[junit4] 2> 3578677 INFO (jetty-closer-3556-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1190e3b{/solr,null,UNAVAILABLE}
[junit4] 2> 3578678 INFO (jetty-closer-3556-thread-2) [ ] o.a.s.c.Overseer Overseer (id=99351359784026121-127.0.0.1:43773_solr-n_0000000000) closing
[junit4] 2> 3578678 INFO (OverseerStateUpdate-99351359784026121-127.0.0.1:43773_solr-n_0000000000) [n:127.0.0.1:43773_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:43773_solr
[junit4] 2> 3578678 WARN (OverseerAutoScalingTriggerThread-99351359784026121-127.0.0.1:43773_solr-n_0000000000) [n:127.0.0.1:43773_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 3578680 INFO (jetty-closer-3556-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@15ee55e{/solr,null,UNAVAILABLE}
[junit4] 2> 3580163 WARN (zkCallback-3601-thread-1-processing-n:127.0.0.1:39905_solr) [n:127.0.0.1:39905_solr ] o.a.s.c.ZkController Unable to register nodeLost path for 127.0.0.1:39905_solr
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/nodeLost/127.0.0.1:39905_solr
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:364)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:364)
[junit4] 2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:879)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:704)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:703)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$900(ZkStateReader.java:72)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1090)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1085)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$1(SolrZkClient.java:268)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 3580166 INFO (jetty-closer-3556-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1d28ce1{/solr,null,UNAVAILABLE}
[junit4] 2> 3580167 ERROR (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 3580167 INFO (SUITE-TestClusterStateProvider-seed#[AD558E4DD12B51C3]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46433 46433
[junit4] 2> 3580302 INFO (Thread-4580) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46433 46433
[junit4] 2> 3580304 WARN (Thread-4580) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 3 /solr/configs/.system
[junit4] 2> 3 /solr/configs/.system/managed-schema
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 23 /solr/collections/.system/state.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 2 /solr/autoscaling.json
[junit4] 2> 2 /solr/overseer_elect/election/99351359784026121-127.0.0.1:43773_solr-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_AD558E4DD12B51C3-001
[junit4] 2> Jan 15, 2018 1:57:32 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@a98a1), locale=sk-SK, timezone=Etc/GMT+6
[junit4] 2> NOTE: Linux 4.13.0-26-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=87225480,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [TestFiltering, TestIntervalFaceting, IndexSchemaTest, TestHashQParserPlugin, TestDynamicFieldResource, TestRetrieveFieldsOptimizer, LeaderElectionTest, CSVRequestHandlerTest, StatsComponentTest, AtomicUpdateProcessorFactoryTest, AnalyticsMergeStrategyTest, TestMaxTokenLenTokenizer, TestFieldSortValues, CoreAdminCreateDiscoverTest, ReturnFieldsTest, HdfsChaosMonkeySafeLeaderTest, CdcrReplicationHandlerTest, HttpSolrCallGetCoreTest, ShardSplitTest, TestFieldCacheSortRandom, TestBackupRepositoryFactory, DistributedSpellCheckComponentTest, BasicDistributedZk2Test, BitVectorTest, TestLRUStatsCache, UnloadDistributedZkTest, TestPushWriter, HdfsDirectoryTest, TestNestedDocsSort, MultiThreadedOCPTest, NodeAddedTriggerTest, TestRestoreCore, MetricsConfigTest, TestConfigSetProperties, PeerSyncReplicationTest, TestUnifiedSolrHighlighter, TestImpersonationWithHadoopAuth, TestSolrCloudWithKerberosAlt, TestTrieFacet, FieldMutatingUpdateProcessorTest, HdfsUnloadDistributedZkTest, SortByFunctionTest, UpdateLogTest, DistributedFacetSimpleRefinementLongTailTest, TestCoreContainer, ScriptEngineTest, SuggesterWFSTTest, DistanceFunctionTest, TestNoOpRegenerator, TestUninvertingReader, AliasIntegrationTest, TestPartialUpdateDeduplication, UUIDFieldTest, TestValueSourceCache, TestIndexSearcher, TestMaxScoreQueryParser, TestPrepRecovery, TestManagedResource, RequiredFieldsTest, TestSolrCoreSnapshots, XmlUpdateRequestHandlerTest, TestNamedUpdateProcessors, TestMacros, HdfsBasicDistributedZk2Test, SolrGraphiteReporterTest, TestReloadDeadlock, IndexBasedSpellCheckerTest, CopyFieldTest, TestLegacyFieldReuse, TestLMJelinekMercerSimilarityFactory, TestAuthenticationFramework, TestSchemalessBufferedUpdates, AlternateDirectoryTest, SuggestComponentContextFilterQueryTest, TestLockTree, ClusterStateTest, SolrCLIZkUtilsTest, BooleanFieldTest, TestExecutePlanAction, TestCoreAdminApis, TestCSVResponseWriter, BlockCacheTest, SuggesterTest, ReplaceNodeTest, TestSearchPerf, TestTrackingShardHandlerFactory, TestCloudManagedSchema, LegacyCloudClusterPropTest, HdfsAutoAddReplicasIntegrationTest, TestNodeAddedTrigger, TestFuzzyAnalyzedSuggestions, LeaderFailureAfterFreshStartTest, UUIDUpdateProcessorFallbackTest, DefaultValueUpdateProcessorTest, TestSolrCoreProperties, TestCloudJSONFacetJoinDomain, TestManagedSynonymFilterFactory, TestRandomFlRTGCloud, TestSolrConfigHandler, TestMinMaxOnMultiValuedField, TestStressInPlaceUpdates, TestCoreDiscovery, TestDistributedGrouping, TestLeaderElectionZkExpiry, CdcrUpdateLogTest, V2ApiIntegrationTest, TestEmbeddedSolrServerConstructors, TestFieldCache, ImplicitSnitchTest, TestDistributedMissingSort, InfoHandlerTest, TestTrie, DirectSolrSpellCheckerTest, TestSolrCLIRunExample, MoveReplicaHDFSFailoverTest, TestSerializedLuceneMatchVersion, HdfsRecoverLeaseTest, SpellCheckCollatorWithCollapseTest, TestSolrConfigHandlerCloud, DateRangeFieldTest, AsyncCallRequestStatusResponseTest, TestReplicaProperties, TestSubQueryTransformer, ChaosMonkeyNothingIsSafeTest, WrapperMergePolicyFactoryTest, HighlighterTest, SystemLogListenerTest, TestClusterStateMutator, TestCloudPseudoReturnFields, TestCloudRecovery, TestConfigSetsAPIExclusivity, TestDeleteCollectionOnDownNodes, TestHdfsCloudBackupRestore, TestPullReplica, TestSolrCloudWithSecureImpersonation, TlogReplayBufferedWhileIndexingTest, TriLevelCompositeIdRoutingTest, ComputePlanActionTest, HttpTriggerListenerTest, TestPolicyCloud, TestClusterStateProvider]
[junit4] Completed [591/767 (1!)] on J2 in 11.13s, 2 tests, 1 failure <<< FAILURES!
[...truncated 44425 lines...]
SOLR-11063 broke TestClusterStateProvider ? -- was: Re: [JENKINS]
Lucene-Solr-master-Linux (32bit/jdk1.8.0_144) - Build # 21274 - Still
Unstable!
Posted by Chris Hostetter <ho...@fucit.org>.
Jira's down for maintence, so i can't report this in jira, but...
These TestClusterStateProvider.testAutoScalingConfig failures seem to reproduce
fairly reliably for me -- across both master and branch_7x. Attempting to
generate new seeds I've never seen this test pass.
Picking one seed at random, and using git bisect starting at when this
test seems to have landed in master points to SOLR-11063 as being the
culprit that broke it...
hossman@tray:~/lucene/alt_dev [master] $ git bisect start HEAD d6d2e3b2e30b600b9f550abf9414743d628cbcf0
Bisecting: 60 revisions left to test after this (roughly 6 steps)
[a56cb42fde41422c99108ff2749c29080bb94e36] LUCENE-8012: LTR contrib needs to use float values in explanations
hossman@tray:~/lucene/alt_dev [detached] $ git bisect run sh -c 'ant clean && cd solr/core/ && ant test -Dtestcase=TestClusterStateProvider -Dtests.method=testAutoScalingConfig -Dtests.seed=893E46D2E59F76F2 -Dtests.slow=true -Dtests.locale=ca-ES -Dtests.timezone=Indian/Cocos -Dtests.asserts=true -Dtests.file.encoding=UTF-8'
running sh -c ant clean && cd solr/core/ && ant test -Dtestcase=TestClusterStateProvider -Dtests.method=testAutoScalingConfig -Dtests.seed=893E46D2E59F76F2 -Dtests.slow=true -D
...
fe86ab982d14b02d5fc9842259f9d0ae1a949757 is the first bad commit
commit fe86ab982d14b02d5fc9842259f9d0ae1a949757
Author: Noble Paul <no...@apache.org>
Date: Fri Jan 12 23:48:30 2018 +1100
SOLR-11063: Suggesters should accept required freedisk as a hint
:040000 040000 b0f02f132700bcf948cc0a9c18d6d6772cdfa792 7a2a8b9242da48cce64003a6f2ff942c2cb8859b M solr
bisect run success
: FAILED: org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig
:
: Error Message:
: expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{ "cluster-preferences":[{"maximize":"freedisk"}], "triggers":{".auto_add_replicas":{ "name":".auto_add_replicas", "event":"nodeLost", "waitFor":30, "actions":[ { "name":"auto_add_replicas_plan", "class":"solr.AutoAddReplicasPlanAction"}, { "name":"execute_plan", "class":"solr.ExecutePlanAction"}], "enabled":true}}, "listeners":{".auto_add_replicas.system":{ "trigger":".auto_add_replicas", "afterAction":[], "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED", "BEFORE_ACTION", "AFTER_ACTION", "IGNORED"], "class":"org.apache.solr.cloud.autoscaling.SystemLogListener", "beforeAction":[]}}, "properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{ "cluster-preferences":[{"maximize":"freedisk"}],
"triggers":{".auto_add_replicas":{ "name":".auto_add_replicas", "event":"nodeLost", "waitFor":30, "actions":[ { "name":"auto_add_replicas_plan", "class":"solr.AutoAddReplicasPlanAction"}, { "name":"execute_plan", "class":"solr.ExecutePlanAction"}], "enabled":true}}, "listeners":{".auto_add_replicas.system":{ "trigger":".auto_add_replicas", "afterAction":[], "stage":[ "STARTED", "ABORTED", "SUCCEEDED", "FAILED", "BEFORE_ACTION", "AFTER_ACTION", "IGNORED"], "class":"org.apache.solr.cloud.autoscaling.SystemLogListener", "beforeAction":[]}}, "properties":{}}>
:
: Stack Trace:
: java.lang.AssertionError: expected: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
: "cluster-preferences":[{"maximize":"freedisk"}],
: "triggers":{".auto_add_replicas":{
: "name":".auto_add_replicas",
: "event":"nodeLost",
: "waitFor":30,
: "actions":[
: {
: "name":"auto_add_replicas_plan",
: "class":"solr.AutoAddReplicasPlanAction"},
: {
: "name":"execute_plan",
: "class":"solr.ExecutePlanAction"}],
: "enabled":true}},
: "listeners":{".auto_add_replicas.system":{
: "trigger":".auto_add_replicas",
: "afterAction":[],
: "stage":[
: "STARTED",
: "ABORTED",
: "SUCCEEDED",
: "FAILED",
: "BEFORE_ACTION",
: "AFTER_ACTION",
: "IGNORED"],
: "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
: "beforeAction":[]}},
: "properties":{}}> but was: org.apache.solr.client.solrj.cloud.autoscaling.AutoScalingConfig<{
: "cluster-preferences":[{"maximize":"freedisk"}],
: "triggers":{".auto_add_replicas":{
: "name":".auto_add_replicas",
: "event":"nodeLost",
: "waitFor":30,
: "actions":[
: {
: "name":"auto_add_replicas_plan",
: "class":"solr.AutoAddReplicasPlanAction"},
: {
: "name":"execute_plan",
: "class":"solr.ExecutePlanAction"}],
: "enabled":true}},
: "listeners":{".auto_add_replicas.system":{
: "trigger":".auto_add_replicas",
: "afterAction":[],
: "stage":[
: "STARTED",
: "ABORTED",
: "SUCCEEDED",
: "FAILED",
: "BEFORE_ACTION",
: "AFTER_ACTION",
: "IGNORED"],
: "class":"org.apache.solr.cloud.autoscaling.SystemLogListener",
: "beforeAction":[]}},
: "properties":{}}>
: at __randomizedtesting.SeedInfo.seed([607345A73B2456EA:5FFB460F2C72A60D]:0)
: at org.junit.Assert.fail(Assert.java:93)
: at org.junit.Assert.failNotEquals(Assert.java:647)
: at org.junit.Assert.assertEquals(Assert.java:128)
: at org.junit.Assert.assertEquals(Assert.java:147)
: at org.apache.solr.cloud.autoscaling.sim.TestClusterStateProvider.testAutoScalingConfig(TestClusterStateProvider.java:214)
: at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
: at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: at java.lang.reflect.Method.invoke(Method.java:498)
: at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
: at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
: at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
: at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
: at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
: at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
: at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
: at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
: at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
: at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
: at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
: at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
: at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
: at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
: at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
: at java.lang.Thread.run(Thread.java:748)
:
:
:
:
: Build Log:
: [...truncated 12344 lines...]
: [junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
: [junit4] 2> 833200 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
: [junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/init-core-data-001
: [junit4] 2> 833201 WARN (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2420 numCloses=2420
: [junit4] 2> 833201 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
: [junit4] 2> 833202 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
: [junit4] 2> 833214 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/tempDir-001
: [junit4] 2> 833214 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
: [junit4] 2> 833214 INFO (Thread-1960) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
: [junit4] 2> 833214 INFO (Thread-1960) [ ] o.a.s.c.ZkTestServer Starting server
: [junit4] 2> 833215 ERROR (Thread-1960) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
: [junit4] 2> 833314 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:45301
: [junit4] 2> 833316 INFO (zkConnectionManagerCallback-1348-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
: [junit4] 2> 833318 INFO (jetty-launcher-1345-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
: [junit4] 2> 833318 INFO (jetty-launcher-1345-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
: [junit4] 2> 833319 INFO (jetty-launcher-1345-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@19b199e{/solr,null,AVAILABLE}
: [junit4] 2> 833319 INFO (jetty-launcher-1345-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7870{/solr,null,AVAILABLE}
: [junit4] 2> 833322 INFO (jetty-launcher-1345-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1c41bd6{SSL,[ssl, http/1.1]}{127.0.0.1:44219}
: [junit4] 2> 833322 INFO (jetty-launcher-1345-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1410f10{SSL,[ssl, http/1.1]}{127.0.0.1:40941}
: [junit4] 2> 833322 INFO (jetty-launcher-1345-thread-1) [ ] o.e.j.s.Server Started @834929ms
: [junit4] 2> 833322 INFO (jetty-launcher-1345-thread-2) [ ] o.e.j.s.Server Started @834929ms
: [junit4] 2> 833322 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44219}
: [junit4] 2> 833322 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=40941}
: [junit4] 2> 833322 ERROR (jetty-launcher-1345-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
: [junit4] 2> 833322 ERROR (jetty-launcher-1345-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
: [junit4] 2> 833323 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
: [junit4] 2> 833323 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
: [junit4] 2> 833323 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
: [junit4] 2> 833323 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
: [junit4] 2> 833323 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
: [junit4] 2> 833323 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
: [junit4] 2> 833323 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-14T20:12:10.827Z
: [junit4] 2> 833323 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2018-01-14T20:12:10.827Z
: [junit4] 2> 833327 INFO (zkConnectionManagerCallback-1351-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
: [junit4] 2> 833327 INFO (zkConnectionManagerCallback-1352-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
: [junit4] 2> 833327 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
: [junit4] 2> 833327 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
: [junit4] 2> 833336 INFO (jetty-launcher-1345-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45301/solr
: [junit4] 2> 833336 INFO (jetty-launcher-1345-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:45301/solr
: [junit4] 2> 833337 INFO (zkConnectionManagerCallback-1358-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
: [junit4] 2> 833337 INFO (zkConnectionManagerCallback-1360-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
: [junit4] 2> 833337 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
: [junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f64d045f0004, likely client has closed socket
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
: [junit4] 2> at java.lang.Thread.run(Thread.java:748)
: [junit4] 2> 833338 INFO (zkConnectionManagerCallback-1363-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
: [junit4] 2> 833338 INFO (zkConnectionManagerCallback-1364-thread-1-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
: [junit4] 2> 833368 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.Overseer Overseer (id=null) closing
: [junit4] 2> 833368 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.Overseer Overseer (id=null) closing
: [junit4] 2> 833369 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:40941_solr
: [junit4] 2> 833369 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44219_solr
: [junit4] 2> 833369 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.Overseer Overseer (id=99350002448400390-127.0.0.1:40941_solr-n_0000000000) starting
: [junit4] 2> 833369 INFO (zkCallback-1362-thread-1-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
: [junit4] 2> 833370 INFO (zkCallback-1361-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
: [junit4] 2> 833373 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:40941_solr
: [junit4] 2> 833374 INFO (zkCallback-1361-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
: [junit4] 2> 833374 INFO (zkCallback-1362-thread-1-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
: [junit4] 2> 833498 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
: [junit4] 2> 833502 WARN (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
: [junit4] 2> 833506 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=localhost.localdomain,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=jenkins, zk-dt-secret-mana
ger.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
: [junit4] 2> 833541 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
: [junit4] 2> 833541 WARN (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
: [junit4] 2> 833541 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=localhost.localdomain,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=jenkins, zk-dt-secret-mana
ger.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
: [junit4] 2> 833562 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
: [junit4] 2> 833562 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
: [junit4] 2> 833576 INFO (jetty-launcher-1345-thread-1-EventThread) [n:127.0.0.1:44219_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
: [junit4] 2> 833576 INFO (jetty-launcher-1345-thread-2-EventThread) [n:127.0.0.1:40941_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
: [junit4] 2> 833644 WARN (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
: [junit4] 2> 833644 WARN (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
: [junit4] 2> 833676 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44219.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
: [junit4] 2> 833677 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40941.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
: [junit4] 2> 833684 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44219.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
: [junit4] 2> 833684 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44219.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
: [junit4] 2> 833685 INFO (jetty-launcher-1345-thread-1) [n:127.0.0.1:44219_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/tempDir-001/node1/.
: [junit4] 2> 833686 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40941.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
: [junit4] 2> 833686 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40941.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
: [junit4] 2> 833687 INFO (jetty-launcher-1345-thread-2) [n:127.0.0.1:40941_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/tempDir-001/node2/.
: [junit4] 2> 833725 INFO (zkConnectionManagerCallback-1370-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
: [junit4] 2> 833735 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
: [junit4] 2> 833784 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noGroups&p=noGroups&t=test&e=1515996731275&s=r5PNXDpWhqFdK6ivqVFRSzQ0JQ8="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:11 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:11 GMT
: [junit4] 2> 833785 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
: [junit4] 2> 833795 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
: [junit4] 2> 839013 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=wrongHost&p=wrongHost&t=test&e=1515996731299&s=Nj/wxDF18s56Os3pCRy1hZaMgNg="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:11 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:11 GMT
: [junit4] 2> 839014 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
: [junit4] 2> 839030 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
: [junit4] 2> 839034 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
: [junit4] 2> 839034 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
: [junit4] 2> 839034 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515996736535&s=GvjxvsvNTp+JnnsjIQ6/3rwppOw="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:16 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:16 GMT
: [junit4] 2> 839034 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
: [junit4] 2> 839044 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
: [junit4] 2> 839046 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noHosts&p=noHosts&t=test&e=1515996736549&s=p7KCygViyWK2/NVnnwUL1sNFQWc="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:16 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:16 GMT
: [junit4] 2> 839047 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
: [junit4] 2> 839056 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
: [junit4] 2> 839056 INFO (qtp9631235-7160) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
: [junit4] 2> 839056 INFO (qtp9631235-7160) [n:127.0.0.1:40941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
: [junit4] 2> 839057 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=test&e=1515996736560&s=EKfjYIxpM5wXFVKn8OjSTYKjT6A="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:16 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:16 GMT
: [junit4] 2> 839057 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
: [junit4] 2> 839065 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
: [junit4] 2> 839069 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515996736569&s=rXAXy6lRq9tjZ/cAhNPgXoN36N0="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:16 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:16 GMT
: [junit4] 2> 839070 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
: [junit4] 2> 839079 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyUnknownRemote
: [junit4] 2> 850729 WARN (jetty-launcher-1345-thread-1-SendThread(127.0.0.1:45301)) [n:127.0.0.1:44219_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17044ms for sessionid 0x160f64d045f0005
: [junit4] 2> 850729 WARN (jetty-launcher-1345-thread-1-SendThread(127.0.0.1:45301)) [n:127.0.0.1:44219_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17088ms for sessionid 0x160f64d045f0008
: [junit4] 2> 850729 WARN (jetty-launcher-1345-thread-2-SendThread(127.0.0.1:45301)) [n:127.0.0.1:40941_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 17088ms for sessionid 0x160f64d045f0007
: [junit4] 2> 850729 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
: [junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f64d045f0008, likely client has closed socket
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
: [junit4] 2> at java.lang.Thread.run(Thread.java:748)
: [junit4] 2> 850730 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
: [junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f64d045f0005, likely client has closed socket
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
: [junit4] 2> at java.lang.Thread.run(Thread.java:748)
: [junit4] 2> 850730 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
: [junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x160f64d045f0007, likely client has closed socket
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
: [junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
: [junit4] 2> at java.lang.Thread.run(Thread.java:748)
: [junit4] 2> 850731 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1515996736583&s=dfmuzR/I2V4b/aD/BcZkvv6i/TU="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:16 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:16 GMT
: [junit4] 2> 850731 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyUnknownRemote
: [junit4] 2> 850743 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
: [junit4] 2> 850754 WARN (qtp9631235-7174) [ ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: ???bar???: no such user
: [junit4] 2> id: ???bar???: no such user
: [junit4] 2>
: [junit4] 2> 850756 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=test&e=1515996748247&s=GPRjGGItE1UyIIwdYDtH4BjZtHw="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:28 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:28 GMT
: [junit4] 2> 850756 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
: [junit4] 2> 850767 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
: [junit4] 2> 850788 INFO (qtp9631235-7160) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
: [junit4] 2> 850788 INFO (qtp9631235-7160) [n:127.0.0.1:40941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&wt=javabin&version=2} status=0 QTime=0
: [junit4] 2> 850788 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostUsersGroup&p=anyHostUsersGroup&t=test&e=1515996748272&s=z9W6+Z/X8M+3wJWuCDoy3HgbeQU="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:28 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:28 GMT
: [junit4] 2> 850788 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
: [junit4] 2> 850799 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
: [junit4] 2> 850801 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1515996748303&s=SbEbEsOm7jevbmKVudIKWskVYVI="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:28 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:28 GMT
: [junit4] 2> 850801 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
: [junit4] 2> 850814 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
: [junit4] 2> 850814 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.s.SolrDispatchFilter Error authenticating
: [junit4] 2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
: [junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
: [junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
: [junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
: [junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
: [junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
: [junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
: [junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
: [junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
: [junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
: [junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
: [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:449)
: [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
: [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
: [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
: [junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
: [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
: [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
: [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
: [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
: [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
: [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
: [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
: [junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
: [junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
: [junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
: [junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
: [junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
: [junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
: [junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
: [junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
: [junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
: [junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
: [junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
: [junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
: [junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
: [junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
: [junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
: [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
: [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
: [junit4] 2> at java.lang.Thread.run(Thread.java:748)
: [junit4] 2> 850815 WARN (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.e.j.s.ServletHandler /solr/admin/collections
: [junit4] 2> org.apache.solr.common.SolrException: Error during request authentication,
: [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:455)
: [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:349)
: [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
: [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
: [junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
: [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
: [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
: [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
: [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
: [junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
: [junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
: [junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
: [junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
: [junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
: [junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
: [junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
: [junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
: [junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
: [junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
: [junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
: [junit4] 2> at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:251)
: [junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
: [junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
: [junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
: [junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
: [junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
: [junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
: [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
: [junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
: [junit4] 2> at java.lang.Thread.run(Thread.java:748)
: [junit4] 2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
: [junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
: [junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
: [junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
: [junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
: [junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
: [junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
: [junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
: [junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
: [junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
: [junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
: [junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:449)
: [junit4] 2> ... 29 more
: [junit4] 2> 850816 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
: [junit4] 2> 850829 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
: [junit4] 2> 850830 INFO (jetty-launcher-1345-thread-1-EventThread) [n:127.0.0.1:44219_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
: [junit4] 2> 850830 WARN (zkConnectionManagerCallback-1363-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ConnectionManager Watcher org.apache.solr.common.cloud.ConnectionManager@11c0caf name: ZooKeeperConnection Watcher:127.0.0.1:45301/solr got event WatchedEvent state:Disconnected type:None path:null path: null type: None
: [junit4] 2> 850830 WARN (zkConnectionManagerCallback-1363-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ConnectionManager zkClient has disconnected
: [junit4] 2> 850835 INFO (zkConnectionManagerCallback-1373-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
: [junit4] 2> 851090 INFO (zkConnectionManagerCallback-1376-thread-1) [ ] o.a.s.c.c.ConnectionManager zkClient has connected
: [junit4] 2> 851091 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
: [junit4] 2> 851092 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:45301/solr ready
: [junit4] 2> 851094 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
: [junit4] 2> 851095 INFO (OverseerThreadFactory-3228-thread-1-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.CreateCollectionCmd Create collection forwardingCollection
: [junit4] 2> 851213 INFO (OverseerStateUpdate-99350002448400390-127.0.0.1:40941_solr-n_0000000000) [n:127.0.0.1:40941_solr ] o.a.s.c.o.SliceMutator createReplica() {
: [junit4] 2> "operation":"ADDREPLICA",
: [junit4] 2> "collection":"forwardingCollection",
: [junit4] 2> "shard":"shard1",
: [junit4] 2> "core":"forwardingCollection_shard1_replica_n1",
: [junit4] 2> "state":"down",
: [junit4] 2> "base_url":"https://127.0.0.1:40941/solr",
: [junit4] 2> "type":"NRT",
: [junit4] 2> "waitForFinalState":"false"}
: [junit4] 2> 851340 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
: [junit4] 2> 851340 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
: [junit4] 2> 851443 INFO (zkCallback-1362-thread-1-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
: [junit4] 2> 851443 INFO (zkCallback-1362-thread-2-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
: [junit4] 2> 851943 INFO (jetty-launcher-1345-thread-2-EventThread) [n:127.0.0.1:40941_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
: [junit4] 2> 852354 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
: [junit4] 2> 852373 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
: [junit4] 2> 852499 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
: [junit4] 2> 852506 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'forwardingCollection_shard1_replica_n1' using configuration from collection forwardingCollection, trusted=true
: [junit4] 2> 852507 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_40941.solr.core.forwardingCollection.shard1.replica_n1' (registry 'solr.core.forwardingCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@cc1628
: [junit4] 2> 852507 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
: [junit4] 2> 852507 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [[forwardingCollection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/tempDir-001/node2/forwardingCollection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_607345A73B2456EA-001/tempDir-001/node2/./forwardingCollection_shard1_replica_n1/data/]
: [junit4] 2> 852510 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=14, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=65.8125, floorSegmentMB=1.486328125, forceMergeDeletesPctAllowed=28.468254921113118, segmentsPerTier=41.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3865847439261596
: [junit4] 2> 852539 INFO (zkConnectionManagerCallback-1363-thread-1-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ConnectionManager zkClient has connected
: [junit4] 2> 852556 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
: [junit4] 2> 852556 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
: [junit4] 2> 852557 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
: [junit4] 2> 852557 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
: [junit4] 2> 852558 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=25, maxMergeAtOnceExplicit=14, maxMergedSegmentMB=94.5517578125, floorSegmentMB=1.3544921875, forceMergeDeletesPctAllowed=17.983388940845256, segmentsPerTier=22.0, maxCFSSegmentSizeMB=1.009765625, noCFSRatio=1.0
: [junit4] 2> 852558 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@663f4a[forwardingCollection_shard1_replica_n1] main]
: [junit4] 2> 852559 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
: [junit4] 2> 852614 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
: [junit4] 2> 852701 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
: [junit4] 2> 852701 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
: [junit4] 2> 852701 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1589600059606958080
: [junit4] 2> 852701 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
: [junit4] 2> 852701 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
: [junit4] 2> 852701 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
: [junit4] 2> 852702 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
: [junit4] 2> 852702 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
: [junit4] 2> 852702 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
: [junit4] 2> 852702 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
: [junit4] 2> 852702 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
: [junit4] 2> 852703 INFO (searcherExecutor-3233-thread-1-processing-n:127.0.0.1:40941_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] Registered new searcher Searcher@663f4a[forwardingCollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
: [junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
: [junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
: [junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:40941/solr/forwardingCollection_shard1_replica_n1/
: [junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
: [junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy https://127.0.0.1:40941/solr/forwardingCollection_shard1_replica_n1/ has no replicas
: [junit4] 2> 852704 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
: [junit4] 2> 852706 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:40941/solr/forwardingCollection_shard1_replica_n1/ shard1
: [junit4] 2> 852808 INFO (zkCallback-1362-thread-2-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
: [junit4] 2> 852808 INFO (zkCallback-1362-thread-3-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
: [junit4] 2> 852857 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
: [junit4] 2> 852859 INFO (qtp9631235-7170) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1519
: [junit4] 2> 852860 WARN (httpShardExecutor-3222-thread-1-processing-n:127.0.0.1:40941_solr [https:////127.0.0.1:40941//solr] https:////127.0.0.1:40941//solr) [n:127.0.0.1:40941_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515996748843&s=ymWWTanFXlfJnkYUi/r+HaMKeVA="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:28 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:28 GMT
: [junit4] 2> 852868 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
: [junit4] 2> 852959 INFO (zkCallback-1362-thread-3-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
: [junit4] 2> 852959 INFO (zkCallback-1362-thread-2-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
: [junit4] 2> 853102 INFO (OverseerCollectionConfigSetProcessor-99350002448400390-127.0.0.1:40941_solr-n_0000000000) [n:127.0.0.1:40941_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
: [junit4] 2> 853868 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=2774
: [junit4] 2> 853869 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515996748597&s=SXUJgIGdLeEuFacohlPCTzBHrDY="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:28 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:28 GMT
: [junit4] 2> 853869 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: forwardingCollection failOnTimeout:true timeout (sec):100
: [junit4] 2> 853869 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: forwardingCollection
: [junit4] 2> 853873 INFO (qtp9631235-7172) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
: [junit4] 2> 853874 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515996751377&s=jWsUk51BbMwaRU7HiGenB8gaeGs="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:31 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:31 GMT
: [junit4] 2> 853919 INFO (qtp2018146-7159) [n:127.0.0.1:44219_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
: [junit4] 2> 853922 INFO (qtp9631235-7162) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
: [junit4] 2> 853923 WARN (qtp2018146-7159) [n:127.0.0.1:44219_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515996751425&s=pt3l9WPnA6CsqnggJLac4bZjslQ="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:31 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:31 GMT
: [junit4] 2> 853927 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1515996751422&s=UXT5V59Toi0WyvJt6tQR2lShGH0="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:31 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:31 GMT
: [junit4] 2> 853927 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1515996751425&s=pt3l9WPnA6CsqnggJLac4bZjslQ="; Path=/; Domain=127.0.0.1; Expires= , 15- -2018 06:12:31 GMT; Secure; HttpOnly". Invalid 'expires' attribute: , 15- -2018 06:12:31 GMT
: [junit4] 2> 853927 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[607345A73B2456EA]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testForwarding
: [junit4] 2> 853928 INFO (jetty-closer-1346-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1c41bd6{SSL,[ssl, http/1.1]}{127.0.0.1:0}
: [junit4] 2> 853928 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=15275865
: [junit4] 2> 853929 INFO (jetty-closer-1346-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1410f10{SSL,[ssl, http/1.1]}{127.0.0.1:0}
: [junit4] 2> 853929 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
: [junit4] 2> 853929 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=8601253
: [junit4] 2> 853929 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@c82e2c: rootName = solr_44219, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@96e270
: [junit4] 2> 853929 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
: [junit4] 2> 853929 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@15fb89c: rootName = solr_40941, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1f144e6
: [junit4] 2> 853933 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
: [junit4] 2> 853933 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
: [junit4] 2> 853933 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@190427d: rootName = solr_44219, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@32710e
: [junit4] 2> 853933 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1b2dff5: rootName = solr_40941, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@32710e
: [junit4] 2> 853936 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
: [junit4] 2> 853936 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7f5d3b: rootName = solr_44219, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@dff10
: [junit4] 2> 853937 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
: [junit4] 2> 853937 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1c4bd73: rootName = solr_40941, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@dff10
: [junit4] 2> 853937 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44219_solr
: [junit4] 2> 853937 INFO (jetty-closer-1346-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
: [junit4] 2> 853938 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40941_solr
: [junit4] 2> 853938 INFO (zkCallback-1361-thread-2-processing-n:127.0.0.1:44219_solr) [n:127.0.0.1:44219_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
: [junit4] 2> 853938 INFO (zkCallback-1362-thread-2-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
: [junit4] 2> 853938 ERROR (Thread[Thread-1977,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:44219_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
: [junit4] 2> 853939 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
: [junit4] 2> 853939 INFO (zkCallback-1362-thread-3-processing-n:127.0.0.1:40941_solr) [n:127.0.0.1:40941_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
: [junit4] 2> 853939 INFO (coreCloseExecutor-3239-thread-1) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@1e5f6f3
: [junit4] 2> 853939 INFO (coreCloseExecutor-3239-thread-1) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.forwardingCollection.shard1.replica_n1, tag=31848179
: [junit4] 2> 853939 INFO (coreCloseExecutor-3239-thread-1) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@13896c: rootName = solr_40941, domain = solr.core.forwardingCollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.forwardingCollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@12ebbf3
: [junit4] 2> 853952 INFO (jetty-closer-1346-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@19b199e{/solr,null,UNAVAILABLE}
: [junit4] 2> 853957 INFO (coreCloseExecutor-3239-thread-1) [n:127.0.0.1:40941_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.forwardingCollection.shard1.leader, tag=31848179
: [junit4] 2> 853958 INFO (jetty-closer-1346-thread-1) [ ] o.a.s.c.Overseer Overseer (id=99350002448400390-127.0.0.1:40941_solr-n_0000000000) closing
: [junit4] 2> 853959 INFO (OverseerStateUpdate-99350002448400390-127.0.0.1:40941_solr-n_0000000000) [n:127.0.0.1:40941_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40941_solr
: [junit4] 2> 853959 WARN (OverseerAutoScalingTriggerThread-99350002448400390-127.0.0.1:40941_solr-n_0000000000) [n:127.0.0.1:40941_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
: [junit4] 2> 853960 ERROR (Thread[Thread-1978,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:40941_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
: [junit4] 2> 853960 INFO (jetty-closer-1346-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@7870{/solr,null,UNAVAILABLE}
: [junit4] 2> 853961 ERROR (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
: [junit4] 2> 853961 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45301 45301
: [junit4] 2> 854183 INFO (Thread-1960) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:45301 45301
: [junit4] 2> 854184 WARN (Thread-1960) [ ] o.a.s.c.ZkTestServer Watch limit violations:
: [junit4] 2> Maximum concurrent create/delete watches above limit:
: [junit4] 2>
: [junit4] 2> 3 /solr/aliases.json
: [junit4] 2> 2 /solr/security.json
: [junit4] 2>
: [junit4] 2> Maximum concurrent data watches above limit:
: [junit4] 2>
: [junit4] 2> 5 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
: [junit4] 2> 5 /solr/collections/forwardingCollection/state.json
: [junit4] 2> 3 /solr/clusterstate.json
: [junit4] 2> 3 /solr/clusterprops.json
: [junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
: [junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
: [junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
: [junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
: [junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMSeqNumRoot
: [junit4] 2>
: [junit4] 2> Maximum concurrent children watches above limit:
: [junit4] 2>
: [junit4] 2> 4 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMTokensRoot
: [junit4] 2> 3 /solr/live_nodes
: [junit4] 2> 3 /solr/collections
: [junit4] 2> 3 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot
: [junit4] 2>
: [junit4] 2> 854194 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[607345A73B2456EA]-worker) [ ] o.a.s.SolrTestCaseJ4 ------------------------------------------------------- Done waiting for tracked resources to be released
: [junit4] 2> Jan 14, 2018 8:12:31 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
: [junit4] 2> WARNING: Will linger awaiting termination of 3 leaked thread(s).
: [junit4] 2> Jan 14, 2018 8:12:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
: [junit4] 2> SEVERE: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
: [junit4] 2> 1) Thread[id=7210, name=jetty-launcher-1345-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
: [junit4] 2> at sun.misc.Unsafe.park(Native Method)
: [junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
: [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
: [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
: [junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
: [junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
: [junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
: [junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
: [junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
: [junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
: [junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
: [junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
: [junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
: [junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
: [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
: [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
: [junit4] 2> 2) Thread[id=7212, name=jetty-launcher-1345-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
: [junit4] 2> at sun.misc.Unsafe.park(Native Method)
: [junit4] 2> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
: [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
: [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
: [junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
: [junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
: [junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
: [junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
: [junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
: [junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
: [junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
: [junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
: [junit4] 2> at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
: [junit4] 2> at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
: [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
: [junit4] 2> at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
: [junit4] 2> Jan 14, 2018 8:12:51 PM com.carrotsearch.randomizedtesting.ThreadLeakControl tryToInterruptAll
: [junit4] 2> INFO: Starting to interrupt leaked threads:
: [junit4] 2> 1) Thread[id=7210, name=jetty-launcher-1345-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
: [junit4] 2> 2) Thread[id=7212, name=jetty-launcher-1345-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
: [junit4] 2> 874293 ERROR (jetty-launcher-1345-thread-2-EventThread) [n:127.0.0.1:40941_solr ] o.a.c.f.i.CuratorFrameworkImpl Watcher exception
: [junit4] 2> java.lang.InterruptedException
: [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1039)
: [junit4] 2> at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
: [junit4] 2> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
: [junit4] 2> at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
: [junit4] 2> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
: [junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
: [junit4] 2> at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
: [junit4] 2> at org.apa
:
: [...truncated too long message...]
:
: xecutor-3556-thread-1) [n:127.0.0.1:42819_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.c.SolrCore [.system_shard1_replica_n2] CLOSING SolrCore org.apache.solr.core.SolrCore@3d8667
: [junit4] 2> 1160127 INFO (coreCloseExecutor-3556-thread-1) [n:127.0.0.1:42819_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core..system.shard1.replica_n2, tag=4032103
: [junit4] 2> 1160128 INFO (coreCloseExecutor-3556-thread-1) [n:127.0.0.1:42819_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@c08e7f: rootName = solr_42819, domain = solr.core..system.shard1.replica_n2, service url = null, agent id = null] for registry solr.core..system.shard1.replica_n2 / com.codahale.metrics.MetricRegistry@1b3127c
: [junit4] 2> 1160131 INFO (jetty-closer-1551-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
: [junit4] 2> 1160131 INFO (jetty-closer-1551-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1138f7d: rootName = solr_40241, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7404be
: [junit4] 2> 1160131 INFO (jetty-closer-1551-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
: [junit4] 2> 1160132 INFO (jetty-closer-1551-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:40241_solr
: [junit4] 2> 1160132 INFO (jetty-closer-1551-thread-3) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@101fa3: rootName = solr_45485, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@7404be
: [junit4] 2> 1160132 INFO (zkCallback-1576-thread-3-processing-n:127.0.0.1:42819_solr) [n:127.0.0.1:42819_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
: [junit4] 2> 1160132 INFO (zkCallback-1572-thread-2-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
: [junit4] 2> 1160133 INFO (zkCallback-1596-thread-1-processing-n:127.0.0.1:45485_solr) [n:127.0.0.1:45485_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
: [junit4] 2> 1160133 INFO (jetty-closer-1551-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
: [junit4] 2> 1160134 INFO (jetty-closer-1551-thread-3) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:45485_solr
: [junit4] 2> 1160135 INFO (jetty-closer-1551-thread-3) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
: [junit4] 2> 1160135 INFO (zkCallback-1576-thread-3-processing-n:127.0.0.1:42819_solr) [n:127.0.0.1:42819_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
: [junit4] 2> 1160135 INFO (zkCallback-1572-thread-2-processing-n:127.0.0.1:40241_solr) [n:127.0.0.1:40241_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
: [junit4] 2> 1160135 INFO (jetty-closer-1551-thread-3) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
: [junit4] 2> 1160135 INFO (zkCallback-1596-thread-1-processing-n:127.0.0.1:45485_solr) [n:127.0.0.1:45485_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
: [junit4] 2> 1160136 INFO (coreCloseExecutor-3557-thread-1) [n:127.0.0.1:40241_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.c.SolrCore [.system_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@92f7f6
: [junit4] 2> 1160137 INFO (coreCloseExecutor-3557-thread-1) [n:127.0.0.1:40241_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core..system.shard1.replica_n1, tag=9631734
: [junit4] 2> 1160137 INFO (coreCloseExecutor-3557-thread-1) [n:127.0.0.1:40241_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@104e79f: rootName = solr_40241, domain = solr.core..system.shard1.replica_n1, service url = null, agent id = null] for registry solr.core..system.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@a9da5f
: [junit4] 2> 1160137 INFO (jetty-closer-1551-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@cd808{/solr,null,UNAVAILABLE}
: [junit4] 2> 1160141 INFO (coreCloseExecutor-3556-thread-1) [n:127.0.0.1:42819_solr c:.system s:shard1 r:core_node5 x:.system_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=4032103
: [junit4] 2> 1160145 INFO (jetty-closer-1551-thread-2) [ ] o.a.s.c.Overseer Overseer (id=99350023575765001-127.0.0.1:42819_solr-n_0000000000) closing
: [junit4] 2> 1160145 INFO (OverseerStateUpdate-99350023575765001-127.0.0.1:42819_solr-n_0000000000) [n:127.0.0.1:42819_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:42819_solr
: [junit4] 2> 1160145 INFO (coreCloseExecutor-3557-thread-1) [n:127.0.0.1:40241_solr c:.system s:shard1 r:core_node3 x:.system_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection..system.shard1.leader, tag=9631734
: [junit4] 2> 1160145 WARN (OverseerAutoScalingTriggerThread-99350023575765001-127.0.0.1:42819_solr-n_0000000000) [n:127.0.0.1:42819_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
: [junit4] 2> 1160147 INFO (jetty-closer-1551-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1a7059b{/solr,null,UNAVAILABLE}
: [junit4] 2> 1161647 INFO (jetty-closer-1551-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1b8b24a{/solr,null,UNAVAILABLE}
: [junit4] 2> 1161648 ERROR (SUITE-TestClusterStateProvider-seed#[607345A73B2456EA]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
: [junit4] 2> 1161648 INFO (SUITE-TestClusterStateProvider-seed#[607345A73B2456EA]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35357 35357
: [junit4] 2> 1176670 INFO (Thread-1472) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:35357 35357
: [junit4] 2> 1176671 WARN (Thread-1472) [ ] o.a.s.c.ZkTestServer Watch limit violations:
: [junit4] 2> Maximum concurrent create/delete watches above limit:
: [junit4] 2>
: [junit4] 2> 5 /solr/aliases.json
: [junit4] 2> 4 /solr/security.json
: [junit4] 2> 3 /solr/configs/.system
: [junit4] 2> 3 /solr/configs/.system/managed-schema
: [junit4] 2>
: [junit4] 2> Maximum concurrent data watches above limit:
: [junit4] 2>
: [junit4] 2> 20 /solr/collections/.system/state.json
: [junit4] 2> 5 /solr/clusterstate.json
: [junit4] 2> 5 /solr/clusterprops.json
: [junit4] 2> 2 /solr/overseer_elect/election/99350023575764999-127.0.0.1:40241_solr-n_0000000001
: [junit4] 2>
: [junit4] 2> Maximum concurrent children watches above limit:
: [junit4] 2>
: [junit4] 2> 5 /solr/live_nodes
: [junit4] 2> 5 /solr/collections
: [junit4] 2>
: [junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-master-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.autoscaling.sim.TestClusterStateProvider_607345A73B2456EA-001
: [junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1858, maxMBSortInHeap=5.144544667996209, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@cd89e5), locale=da, timezone=Pacific/Truk
: [junit4] 2> NOTE: Linux 4.13.0-26-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=237171552,total=518979584
: [junit4] 2> NOTE: All tests run in this JVM: [TestTolerantUpdateProcessorCloud, TestSolrFieldCacheBean, TestFieldCacheSortRandom, DistribJoinFromCollectionTest, ExitableDirectoryReaderTest, UniqFieldsUpdateProcessorFactoryTest, TestSolrConfigHandler, FullSolrCloudDistribCmdsTest, TestExtendedDismaxParser, CdcrReplicationHandlerTest, DistributedFacetSimpleRefinementLongTailTest, TestExclusionRuleCollectionAccess, TestFieldSortValues, TestFunctionQuery, TestCoreDiscovery, MultiThreadedOCPTest, TestDistributedGrouping, TestSolrDeletionPolicy2, DistributedExpandComponentTest, ReplaceNodeTest, TestJsonRequest, TestRestoreCore, TestSolrCloudWithKerberosAlt, TestAuthenticationFramework, AliasIntegrationTest, TestReplicaProperties, TestPerFieldSimilarityWithDefaultOverride, LargeFieldTest, CdcrUpdateLogTest, TestNestedDocsSort, FieldMutatingUpdateProcessorTest, TestTrieFacet, TestCollectionAPI, ShardSplitTest, DocValuesNotIndexedTest, PreAnalyzedUpdateProcessorTest, NodeAddedTriggerTe
st, TestManagedSchemaThreadSafety, MetricsConfigTest, UnloadDistributedZkTest, TestImpersonationWithHadoopAuth, TestPartialUpdateDeduplication, TestManagedSynonymFilterFactory, SolrCoreCheckLockOnStartupTest, TestMacros, TestCloudJSONFacetJoinDomain, TestBackupRepositoryFactory, BooleanFieldTest, TestDeleteCollectionOnDownNodes, TestLFUCache, CSVRequestHandlerTest, HighlighterTest, IndexBasedSpellCheckerTest, TestReload, TestStressInPlaceUpdates, TestClusterStateProvider]
: [junit4] Completed [274/767 (2!)] on J2 in 21.09s, 2 tests, 1 failure <<< FAILURES!
:
: [...truncated 45454 lines...]
:
-Hoss
http://www.lucidworks.com/